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Abstract 

This  thesis  covers  three  objectives:  1)  a  background 
reference  document  concerning  recoverable  spares  cost 
estimating  was  developed;  2)  a  representative  sample  of 
existing  spares  cost  models  were  evaluated;  and  3)  aircraft 
physical  and  performance  characteristics  were  used  to 
develop  a  model  for  estimating  annual  replenishment  spares 
costs.  The  two-part  model  involved  developing  a 
condemnations  cost  estimating  relationship  (CER)  first  and 
then  developing  both  a  CER  and  spreadsheet  generated  factors 
which  related  condemnations  to  replenishment  spares  costs. 

Pour  condemnation  CERs  were  evaluated--one  linear,  one 
arithmetic  transformation  and  two  logarithmic  transformation 
(both  X  and  Y)  CERs.  Only  the  logarithmic  transformations 
provided  statiscally  acceptable  results,  and  even  these 
models  exhibited  wide  prediction  intervals.  This  weakness 
was  due  to  the  large  amount  of  variability  in  the  CER 
databases  (as  evidenced  by  the  number  of  outliers). 

The  CER  relating  condemnations  to  replenishment  spares 
costs  was  a  poorer  statistical  performer.  Spreadsheet 
generated  factors  showed  that  the  ratio  of  replenishment 
spares  to  condemnations  requirements  exhibited  a  downward 
trend  across  the  data  years--suggesting  that  the  factors 
should  be  periodically  re-validated.  Mission  design 
averages  for  fourteen  weapon  systems  are  also  provided. 
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A  MODEL  FOR  ESTIMATING  AIRCRAFT  RECOVERABLE 


SPARES  ANNUAL  COSTS 


I.  Introduction 


General  Issue 

In  the  not  too  distant  past,  when  new  aircraft  were 
being  considered  for  development  and  production,  logistics 
support  "took  a  back  seat"  to  considerations  such  as 
acquisition  cost,  schedule,  and  performance  (Tempi in: 21-22 ) . 
Current  Air  Force  policy  now  recognizes  the  need  to  consider 
logistical  support  throughout  a  weapon  system’s  life  cycle, 
from  the  earliest  stage  of  concept  exploration  to  final 
phase-out  and  disposal  (TASC,  1989 :Ch  5,  3).  This  change  in 
emphasis  is  appropriate  because  operations  and  support  (O&S) 
costs,  at  60%,  typically  account  for  the  largest  share  of  a 
aircraft's  total  life  cycle  cost  (TASC,  1989:Ch  5,  9). 

While  political  pressure  is  still  great  to  reduce  the 
acquisition  costs  of  new  aircraft,  an  underlying  theme  of 
this  new  emphasis  on  logistics  support  is  that  by  spending 
more  money  up  front  to  buy  highly  reliable/maintainable 
weapon  systems,  the  Air  Force  can  save  money  over  the 
system's  life  cycle  due  to  reduced  O&S  costs. 

A  major  contributor  to  these  O&S  cost  is  the  cost  of 
recoverable  spares  (those  which  can  be  repaired  when 
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broken) .  Several  factors  point  to  the  need  for  improved 
spares  cost  prediction  capabilities. 

The  first  factor  is  the  inadequacy  of  current  cost 
models  which  can  be  used  early  in  a  weapon  system's  life 
cycle.  Because  cost  estimates  must  be  created  for  new 
weapon  systems  long  before  detailed  maintenance  plans  and 
aircraft  performance  history  are  available,  cost  models  used 
in  the  early  design  stage  typically  have  limited  input  data 
and  provide  estimates  only  at  highly  aggregated  levels  (May, 
1982:Ch  3,  5-6;  TASC,  1989:Ch  5,  55).  Cost  estimating 
relationship  (CER)-based  models  (also  known  as  parametric 
models)  are  typically  used  during  this  stage.  Even  this 
early  in  the  acquisition  life  cycle,  however,  important 
information  concerning  such  factors  as  aircraft  physical  and 
performance  characteristics  are  often  evaluated  for  their 
cost  impact.  The  problem  lies  in  the  fact  that, 
historically,  parametric  spares  models  used  at  this  stage 
have  failed  to  include  these  important  potential  cost 
drivers  (May,  1982:Ch  3,  3;  Rexroad,  Tillia,  and  Tritle, 
1990:1).  Because  the  models  do  not  adequately  explain  what 
logically  "drives"  their  estimates,  they  are  difficult  to 
defend.  Additionally,  since  they  are  typically  based  solely 
on  acquisition  costs,  any  increases  in  design  or  production 
will  indicate  increased  O&S  costs.  Thus,  they  can't  be  used 
to  justify  additional  acquisition  funds  to  save  O&S  funds. 
They  provide  inadequate  insight  into  how  cost  changes  in 
relation  to  the  various  design  tradeoffs  being  evaluated. 


2 


Cost  estimates  are  also  required  to  support  the 
development  of  spares  budgets  several  years  before  the 
actual  deployment  of  new  weapon  systems.  The  same  situation 
described  above  applies  here  as  well.  The  parametric  cost 
models  used  to  support  the  budget  requirements  provide  no 
insight  into  what  is  driving  the  estimate.  The 
requirements,  therefore,  are  hard  to  defend  and  are  subject 
to  Congressional  budget  cuts. 

Another  cause  for  concern  about  current  spares  cost 
prediction  capabilities  is  that  HQ  AFLC/FMBSR  has  recently 
transferred  most  of  its  responsibility  for  initial  spares 
budget  preparation  to  the  Air  Logistics  Centers  (ALCs) 

[FMBSR  still  budgets  for  common  support  equipment,  a  small 
percentage  of  the  total  requirement,  and  whole  engine  spares 
are  handled  separately].  The  FY  92/93  Budget  Estimate 
Submission  (BES),  dated  September  1990,  marked  the  first 
time  that  the  ALCs  were  given  this  responsibility.  Faced 
with  this  new  task,  the  ALCs  had  many  procedural  questions 
for  the  previous  estimators--HQ  AFLC/FMBSR  (Neuhart,  1991). 
Given  the  fact  that  FMBSR  has  been  using  a  technique 
criticized  for  its  exclusion  of  cost  drivers  (as  described 
above),  new  techniques  are  desired  (Rexroad,  Tillia,  and 
Tritle,  1990:1). 

A  final  factor  is  the  Defense  Management  Review 
Decision  (DMRD)  904,  dated  9  November  1989,  which  placed 
depot  level  repairable  parts  under  a  stock  fund  concept 
(LaGrone,  1990:Ch  1,  1).  Under  this  concept,  control  of 
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obligation  authority  for  recoverable  spares  will  also  be 
handed  down  from  HQ  AFLC  to  the  individual  ALCs .  This 
should  prompt  much  greater  interest  from  the  ALCs  in 
obtaining  new,  more  defendable  cost  estimating  models  for 
use,  not  only  in  the  early  acquisition  stages,  but  over  the 
entire  life  cycle  of  their  weapon  systems. 

Specific  Problem 

A  much  criticized,  yet  common  CER  used  to  predict 
spares  costs  involves  multiplying  recurring  aircraft  flyaway 
costs  by  a  spares  "factor"  (Dement,  1990; Sec  1;  May, 
1982:Chap  3,3;  Rexroad,  Tillia,  and  Tritle,  1990:1).  The 
underlying  logic  of  this  model  is  limited  to  the  assumption 
that  the  greater  a  weapon  system's  flyaway  cost,  the  more 
expensive  its  spares  will  be.  Potential  cost  drivers  such 
as  aircraft  physical  and  performance  characteristics  are 
excluded.  Parametric  models  which  provide  greater  insight 
into  underlying  causal  relationships  are  needed. 

In  addition  to  needing  improved  models  for  spares  cost 
estimating,  both  DMRD  904  and  the  ALCs’  new  responsibility 
for  estimating  spares  costs  necessitate  a  greater 
understanding  of  the  entire  spares  cost  estimating  process 
at  management  levels  below  the  major  command.  As  new 
personnel  become  involved  in  estimating  spares  costs  for  the 
first  time,  they  will  need  a  basic  understanding  of  the 
factors  they  must  consider  and  a  familiarity  with  the 
current  estimating  techniques  available.  Although  numerous 
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O&S  cost  models  can  be  used  to  predict  recoveraole  spares 
requirements,  frequently  organizations  are  familiar  with 
their  own  model (s)  but  know  little,  if  anything,  about  other 
available  models.  Research  is  needed  to  differentiate 
between  the  models  and  identify  their  appropriate  uses. 


Research  Objectives 

Given  these  serious  deficiencies  in  the  current 
recoverable  spares  cost  estimating  field,  the  objectives  of 
this  thesis  are  threefold: 

1)  A  general  overview  of  the  entire  recoverable  spares 
cost  estimating  process  will  be  provided  for 
personnel  new  to  this  field. 

2)  A  summary-level  description  of  several  currently 
available  cost  models  a«.'plicable  to  recoverable 
spares  will  be  provided. 

3)  Aircraft  physical  and  performance  characteristics 
will  be  evaluated  as  po'"ential  cost  drivers  to 
develop  a  new  parametrif  model  for  estimating 
annual  replenishment  spares  costs. 


In  satisfying  these  objectives  there  are  a  number  of 
questions  that  will  have  to  be  answered: 

1)  How  are  recoverable  spares  currently  managed? 

2)  What  factors  afiect  the  cost  of  recoverable  spares? 

3)  How  do  cost  estimarlno  techniques  vary  depending  on 
the  estimate's  purpose  and/.'ii.  one  aircraft's  life 
cycle  stage? 

4)  What  are  the  purposes,  algorithms,  data  inputs  and 
sources,  and  underlying  logic  of  the  existing 
models? 
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Scope 

The  scope  of  this  paper  is  limited  to  estimating  the 
cost  of  recoverable  spares.  These  spares  can  be  defined  as 
"repairable  parts,  assemblies.  *.%nponents ,  etc,  used  in  the 
repair  of  higher  level  a?.'- 4 '  (Reynolds,  1989:49). 

This  excludes  repair  parts,  ,ch  can  be  defined  as 
"consumable,  non-repai-rable  c-s.-!  used  to  repair  higher 
level  assemblies"  (Reynolds,  l;>.. :  49 ) .  The  following 
sentence  uses  these  terms  in  context:  "It  may  be  cheaper  to 
repair  a  broken  'spare'  circuit  board  than  to  buy  a  new  one, 
but  it  is  never  more  economical  to  fix  a  broken  'repair 
part'  such  as  a  nut  or  a  screw." 

Additionally,  the  costs  estimated  are  limited  to 
peacetime  operating  stock  (POS)  requirements  and  therefore 
exclude  War  Readiness  Material  (WRM)  calculations.  The 
logistics  support  J  equired  during  war  time  is  very  different 
than  that  needed  in  peace  time  and  therefore  the  two 
scenarios  require  separate  cost  prediction  models 
(Hoffmayer,  Finnegan,  Jr.,  and  Rogers,  1980:5). 

According  to  reliability  theory  (Gill,  1991:21-41),  the 
number  of  component  failures  (and  therefore  condemnations) 
is  higher  early  in  a  weapon  system's  life  cycle  due  to 
manufacturing  defects  and  higher  later  ia  the  weapon 
system's  life  cycle  due  to  wear  out.  This  thesis  is  limited 
to  predicting  recoverable  spares  requirements  during  the 
time  between  these  two  extremes--when  failure  rates  are, 
according  to  theory,  fairly  constant.  In  order  to 
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II .  Background 


Introduction 

To  enhance  readability,  the  general  spares  cost 
estimating  overview  and  summary  of  existing  cost  models 
applicable  to  spares  (research  objectives  one  and  two)  are 
included  as  Appendix  B  and  C  to  the  thesis.  Those  readers 
who  are  new  to  spares  cost  estimating  may  want  to  read  these 
appendices  before  proceeding.  This  chapter  relates  to  the 
third  research  ob jective--development  of  a  new  spares  CER. 

It  begins  by  describing  the  selection  of  data  used  in  the 
CER  development.  Following  this,  a  brief  justification  for 
the  use  of  the  parametric  estimating  technique  is  provided 
and  the  assumptions  of  linear  regression  are  addressed. 

Next,  existing  spares  CERs  are  evaluated  in  an  attempt  to 
narrow  down  the  list  of  cost  driver  candidates.  Finally,  a 
brief  summary  of  the  literature  review  is  provided. 

Selection 

Perhaps  one  of  the  most  difficult  aspects  of  developing 
a  good  aircraft  spares  CER  is  obtaining  an  adequate 
database.  This  section  begins  by  describing  the  problems 
associated  with  using  obligation  data  as  a  dependent 
variable.  Condemnation  spares  costs  were  chosen  as  an 
alternative  dependent  variable  for  a  replenishment  spares 
model  and  the  rationale  for  this  decision  is  discussed. 
Because  condemnation  costs  do  not  equate  exactly  to 
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(called  the  dependent  variable),  to  one  or  more  other 
cost  drivers  (called  the  independent  variables)  (Ch  10, 
3). 

Cost  Drivers  -  "Those  [independent]  variables  that 
exhibit  some  systematic  relationship  with  cost"  (Ch  10, 
3). 

Flyaway  Costs  -  Non-recurring  plus  recurring  costs 
for  airframe,  propulsion  and  avionics,  program 
management,  test  and  evaluation,  [and]  allowances  for 
engineering  changes.  (Levine  and  Horowitz,  1989:5) 

Initial  Spares  -  Repairable  components  which  support 
newly  fielded  end  items  (or  principal  items)  for  the 
entire  production  run  of  the  aircraft  (Rexroad,  Tillia, 
and  Tritle,  1990:21). 

Life  Cycle  Cost  (LCC)  -  The  total  cost  to  the 
Government  of  acquisition  and  ownership  of  the  system 
over  its  full  life  It  includes  the  cost  of 
development,  acquisition,  operation,  support,  and  where 
applicable,  disposal.  (Ch  5,  33) 

Obligation  -  As  used  in  the  Air  Force  program  control 
community,  funds  are  said  to  be  "obligated"  at  that 
point  in  time  when  the  contractual  agreement  between 
the  Air  Force  and  contractor  is  posted  into  the 
official  accounting  and  finance  records.  "Obligations" 
are  separate  from  "commitments"  and  "expenditures." 

The  former  takes  place  when  a  purcnase  request  or  other 
authorized  commitment  document  is  signed  by  the 
accounting  and  finance  certifying  official  and  the 
latter  takes  place  when  the  Air  Force  actually  pays  the 
Contractor . 

Operations  and  Support  (O&S)  Costs  -  Fixed  and  variable 
costs  of  personnel,  material,  facilities,  and  other 
items  needed  largely  for  the  peacetime  operation, 
maintenance,  and  support  of  a  system  during  activation, 
steady  state  operation,  and  disposal.  (Ch  17,  3) 

Provisioning  -  The  process  of  determining  and  acquiring 
the  range  and  quantity  (depth)  of  spares  and  repair 
parts,  and  support  and  test  equ’,p..ient  required  to 
operate  and  maintain  an  end  item  of  material  for  an 
initial  period  of  service.  (Ch  5,  34) 

Reliability  -  The  probability  that  the  system  will 
satisfy  the  need  for  which  it  was  intended  in  an 
acceptable  manner,  for  a  given  period  of  time,  when 
deployed  end  used  under  a  given  set  of  operating 
conditions.  .  .  .  Satisfactory  performance  describes 


8 


the  level  at  which  the  item/system  must  perform; 
performance  below  this  level  is  then  considered 
"failure"  even  though  the  specific  part/component  has 
not  broken  or  reached  a  zero  performance  level.  (Ch  5, 
16) 

Replenishment  Spares  -  Repairable  components, 
assemblies,  or  subassemblies  required  to  resupply 
initial  stockage  or  increased  stockage  for  reasons 
other  than  support  of  newly  fielded  end  items. 
Replenishment  would  include  additional  stockage  due  to 
increases  such  as  usage,  readiness  initiatives,  and 
redeployment  of  end  items.  (Ch  17,  40) 
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To  enhance  readability,  the  general  spares  cost 
estimating  overview  and  summary  o£  existing  cost  models 
applicable  to  spares  (research  objectives  one  and  two)  are 
included  as  Appendix  B  and  C  to  the  thesis.  Those  readers 
who  are  new  to  spares  cost  estimating  may  want  to  read  these 
appendices  before  proceeding.  This  chapter  relates  to  the 
third  research  ob jective--devel opment  of  a  new  spares  CER. 

It  begins  by  describing  the  selection  of  data  used  in  the 
CER  development.  Following  this,  a  brief  justification  for 
the  use  of  the  parametric  estimating  technique  is  provided 
and  the  assumptions  of  linear  regression  are  addressed. 

Next,  existing  spares  CERs  are  evaluated  in  an  attempt  to 
narrow  down  the  list  of  cost  driver  candidates.  Finally,  a 
brief  summary  of  the  literature  review  is  provided. 
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Perhaps  one  of  the  most  difficult  aspects  of  developing 
a  good  aircraft  spares  CER  is  obtaining  an  adequate 
database.  This  section  begins  by  describing  the  problems 
associated  with  using  obligation  data  as  a  dependent 
variable.  Condemnation  spares  costs  were  chosen  as  an 
alternative  dependent  variable  for  a  replenishment  spares 
model  and  the  rationale  for  this  decision  is  discussed. 
Because  condemnation  costs  do  not  equate  exactly  to 
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replenishment  spares  costs,  the  findings  from  previous 
studies  of  the  relationship  between  these  two  cost 
categories  is  provided.  Next,  the  problems  associated  with 
finding  an  alternative  dependent  variable  for  an  initial 
spares  model  are  discussed.  Finally,  the  logic  behind  the 
independent  variable  candidates  analyzed  in  the  CER 
development  is  provided. 

Dependent  Variable  Data  Problem.  The  problem  in  spares 
cost  model  development  most  frequently  mentioned  in  the 
literature  is  the  lack  of  good  data  to  validate  the  cost 
models  (Alexander,  Brookey,  Erhart,  Fulton,  Hofmann,  and 
Shutak,  1990:ch  IX,  7;  Dement,  1991;  Levine  and  Horowitz, 
1989:1;  May,  1982:Ch  10,  5-6).  The  most  commonly  used 
dependant  variable  for  spares  cost  model  development  was 
actual  spares  obligations.  Obligation  data  presents  two 
general  problems:  1)  reliable  data  is  hard  to  find,  and  2) 
even  if  reliable  data  can  be  found,  what  "was  obligated"  is 
still  a  poor  proxy  for  what  "should  have  been  obligated." 

There  were  several  reasons  given  for  why  finding 
reliable  obligation  data  is  judged  "difficult  to  impossible" 
(Alexander,  Brookey,  Erhart,  Fulton,  Hofmann,  and  Shutak, 
1990 :Ch  IX,  7).  The  current  definition  of  initial  spares 
(Rexroad,  Tillia,  and  Tritle,  1990:21)  has  only  been  in 
existence  since  March  1985  when  the  Office  of  the  Secretary 
of  Defense  provided  the  current  interpretation.  Until  that 
time  there  was  no  formal  definition.  The  common  practice. 
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however,  was  to  budget  enough  initial  spares  money  (BP16)  to 
cover  the  first  two  years'  requirements  after  initial 
deployment  of  the  weapon  system;  with  subsequent  spares 
funded  by  replenishment  spares  money.  Because  initial 
spares,  as  currently  defined,  now  cover  the  entire 
production  run  of  the  weapon  system,  including  any  increased 
requirements  for  previously  fielded  systems  (Neuhart,  1990), 
the  mix  of  initial  and  replenishment  spares  funding  has 
shifted.  This  definitional  change  makes  it  difficult  to 
separate  weapon  system  obligations  over  time  between  initial 
spares  and  replenishment  spares. 

Additional  reasons  cited  for  difficulty  in  finding 
reliable  obligation  data  include: 

vast  quantities  of  data  at  very  detailed  levels  with 
few  pre-existing  levels  of  data  aggregation  other  than 
the  top  ones;  [and] 

separation  of  explanatory  worksheet  files  from  the 
quantitative  data.  .  .  .  (Alexander,  Brookey,  Erhart, 
Fulton,  Hofmann,  and  Shutak,  1990 :ch  IX,  7) 

Once  these  difficulties  are  overcome  one  is  still  faced 
with  the  fact  that  "what  was  obligated"  does  not  necessarily 
coincide  with  "what  should  have  been  obligated."  Many  cost 
model  projections  are  based  upon  given  values  for 
independent  variables  such  as  the  number  of  flying  hours, 
maintenance  factors,  and  part  utilization  rates  (Rexroad, 
Tillia,  and  Tritle,  1990:7,  9,  11,  12,  14).  Through  these 
factors  the  cost  estimate  is,  in  effect,  provided  for  and 
based  upon  a  given  demand/ avail ability  performance  level. 
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However,  flexibility  is  allowed  and  the  "replenishment 
spares  budget  does  not  have  to  be  expended  in  the  same 
manner  as  it  is  justified"  (May,  1982:  ch  10,  6).  The 
operational  performance  level  used  as  the  basis  for  the  cost 
estimate  may  therefore  not  be  achieved.  The  cost  estimate 
could  conceivably  provide  accurate  estimates  for  the 
proposed  performance  level  and  still  not  resemble  the  actual 
obligation  figures  because  of  the  difference  between  the 
proposed  and  achieved  performance  levels.  In  addition, 
switching  obligations  from  one  weapon  system  to  another  can 
be  influenced  by  political  considerations  not  envisioned  in 
the  cost  model.  Thus,  the  obligations'  correlation  to  the 
cost  estimate  is  greatly  impaired  (Dement,  1991). 

Finally,  it  is  hard  to  track  obligations  to  actual 
usage  of  spares  because:  1)  the  spares  budgets  are  forecast 
two  years  before  funds  are  actually  obligated  (due  to  budget 
cycle)  for  spares  that  won't  be  used  for  another  two  years 
(due  to  contractor  delivery  lead  time)  and  2)  the  obligation 
money  budgeted  for  one  fiscal  year,  say  1990,  can  be  spent 
over  three  fiscal  years  (1990  -  1992)  making  it  impossible 
to  determine  what  spares  are  procured  with  what  budget 
year's  obligations  (May,  1982;  ch  10,  6)  [Note  that  spares 
money  will  only  be  good  for  one  year's  obligation  under  the 
new  stock  fund  concept  but  this  problem  still  affects  old 
data] . 

The  end  result  of  these  data  problems  is  that:  1)  those 
cost  models  developed  using  obligation  data  as  their 
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dependent  variable  impair  the  internal  validity  of  their 
model  and  must  Co.  at  their  results,  or  2)  cost  models  are 
developed  based  on  logical  relationships  but  never  validated 
against  actual  data. 

Initial  Spares  Data .  Because  of  these  problems  with 
obligation  data,  an  attempt  was  made  to  identify  another 
suitable  dependent  variable.  Historically,  differences  in 
the  requirements  development  processes  for  initial  and 
replenishment  spares  has  made  alternative  dependent  variable 
identification  more  difficult  for  initial  spares. 

In  the  replenishment  spares  arena,  annual  buy 
requirements  are  generated  by  the  D041,  "Recoverable 
Consumption  Item  Requirement  System."  These  near  term 
requirements  are  then  used  as  inputs  to  a  regression-based 
model  known  as  the  Air  Logistics  Early  Requirements 
Technique  (ALERT)  which  is  used  to  develop  Program  Objective 
Memorandum  (POM)  requirements  [the  POM  is  a  long  range 
budget  document].  This  computerized  process  provides  a 
central  database  of  historical  requirements  that  can  be  used 
as  an  alternative  dependent  variable  data  set. 

The  situation  is  different  in  the  initial  spares  arena. 
Because  the  initial  spares  requirements  process,  governed  by 
AFLCR  57-27,  has  to  date  been  primarily  a  manual  system,  no 
central  database  of  annual  initial  spares  requirements 
exist.  Without  a  central  database,  no  methodology  akin  to 
ALERT  has  been  developed  for  budget  development  purposes. 
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To  develop  a  historical  database,  one  would  have  to 
contact  the  numerous  inventory  management  specialists  and 
end  article  item  managers  at  each  of  the  Air  Logistics 
Centers.  It  is  these  individuals  who  developed  the  initial 
spares  requirements  and,  hopefully,  kept  records  of  their 
manual  computations. 

A  computerized  approach  to  developing  initial  spares 
requirements  using  computations  similar  to  D041  is  currently 
in  systems  validation  test  and  is  due  to  come  on  line  by  the 
end  of  the  summer  of  1991  (Horner,  1991).  The  new 
computerized  system,  known  as  the  Initial  Requirements 
Determination  (IRD)  system,  will  have  the  capability  to 
store  historical  data  and  will  be  useful  to  future  analysts. 
Given  the  time  constraints  of  this  thesis,  this  task  was  not 
attempted  and  therefore  a  CER  for  initial  spares  is  not 
attempted. 

Replenishment  Spares  Data .  The  central  requirements 
database  for  replenishment  spares  made  identifying  an 
alternative  dependent  variable  an  easier  task.  A  partial 
solution  to  the  problem  was  found  in  "condemnation  spares" 
costs.  Current  AF  policy  states  that  if  the  cost  to  repair 
a  failed  spare  part  exceeds  75%  of  the  cost  to  purchase  a 
new  spare,  the  item  is  "condemned"  (i.e.,  not  repaired) 
(Novak,  1991).  Condemnation  spares,  therefore,  are  those 
new  spares  purchased  to  replace  the  condemned  spares. 

Condemnation  spares  make  for  a  better  dependent 
variable  in  that  they  are  not  obfuscated  by  many  of  the 
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factors  affecting  total  replenishment  spares  obligations. 

For  example,  an  aircraft's  annual  condemnation  spares  costs 
(as  maintained  in  the  Weapon  System  Cost  Retrieval  System 
(WSCRS))  are  the  aggregated  product  of  all  its  condemned 
parts  multiplied  by  the  latest  purchase  price  on  record  for 
these  parts.  Although  condemnations  are  only  a  part  of  the 
total  replenishment  spares  requirement,  they  are,  at  least, 
a  true  indicator  of  spares  usage.  Additionally,  the  WSCRS 
condemnation  costs  for  each  fiscal  year  (FY)  correspond 
directly  to  the  condemnations  requirements  for  the  same  FYs . 
Finally,  condemnations  spares  requirements  track  logically 
to  cost  drivers  such  as  aircraft  performance  and  physical 
characteristics . 

Alternatively,  obligations  take  place  acquisition  lead 
time  away  (approximately  two  years)  from  the  actual  demand 
and  are,  therefore,  only  estimates  of  the  true  demand. 
Obligations  are  tracked  by  fiscal  years  just  like 
condemnations;  but,  as  mentioned  before,  because  any  given 
fiscal  year's  funds  may  be  obligated  over  a  three  year  time- 
frame,  the  obligations  are  much  harder  to  track  to  the 
spares  they  purchase.  Finally,  like  condemnations, 
obligations  may  logically  track  to  aircraft  performance  and 
physical  characteristics,  but  they  are  greatly  influenced  by 
other  factors,  such  as  political  pressures,  which  aren't 
accounted  for  in  the  model  (Dement,  1991). 

It  was  stated  that  condemnation  costs  are  just  a 
partial  solution  to  the  replenishment  spares  dependent 
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variable  data  set  problem.  This  is  true  because 
condemnations  costs  are  just  one  segment  of  total 
replenishment  spares  costs.  In  a  world  with  perfect 
predictive  powers,  the  term  "replenishment"  spares  would  be 
completely  accurate  because  all  the  factors  affecting  demand 
for  spares  would  be  known  (e.g.,  flying  hour  programs,  parts 
utilization  rates,  maintenance  factors).  Because  (in  this 
perfect  world)  the  right  number  of  initial  spares  would 
always  be  bought,  the  replenishment  spares  function  would 
simply  be  to  "replenish"  those  spares  which  are  no  longer 
economically  repairable  (condemnations).  In  reality,  funds 
aren't  always  available  to  purchase  the  desired  number  of 
initial  spares,  mission  requirements  change,  and  actual 
performance  histories  show  that  maintenance  factors  need 
revision.  The  result  is  that  sometimes  the  Air  Force  buys 
too  many  initial  spares,  sometimes  too  few.  At  other  times, 
changes  require  the  purchase  of  spares  for  other  than 
replacement  of  condemnations. 

Demand  Volatility.  All  of  these  factors  which 
make  up  the  difference  between  condemnation  spares  costs  and 
total  replenishment  spares  costs  drive  what  is  known  in  the 
HQ  AFLC/PMCA  office  as  the  "churn"  or  "demand  volatility" 
factor.  It  (demand  volatility)  accounts  for  the  fact  that 
replenishment  spares  funds  are  used  to  purchase  additive 
requirements  and  (on  occasion)  pipeline  spares  requirements. 
The  "churn"  factor  used  in  FMCA’s  Logistics  Support  Cost 
(LSC)  model  is  2.15  (Novak,  1991).  This  value  is  multiplied 
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by  the  condenmation  spares  cost  generated  by  the  model  to 
arrive  at  the  total  replenishment  spares  cost. 

The  condemnation  costs  were  identified  as  being  a 
partial  solution  to  the  dependent  data  set  problem  because 
demand  volatility  must  be  accounted  for  before  a  total 
replenishment  spares  cost  can  be  predicted.  To  complete  the 
spares  models  for  this  thesis,  research  will  be  done  to 
either  validate  or  improve  upon  the  current  churn  factor. 

Two  churn  studies  were  evaluated. 

Mr.  Bob  Novak,  a  Operations  Research  Analyst  from  HQ 
AFLC/FMCA,  performed  the  first  of  the  analyses  in  1987  using 
replenishment  spares  cost  predictions  obtained  from  the  D041 
computer  system.  The  D041  product  used  in  budget 
development  is  known  as  the  Central  Secondary  Item 
Stratification  (CSIS)  report.  Each  June  CSIS  contains  spare 
requirements  forecasts  for  the  June  quarter  and  the 
following  12  quarters.  As  the  fiscal  year  progresses,  each 
passing  quarter  is  dropped  from  the  computations  until  the 
following  June  cycle  when  another  fiscal  year's  (four 
quarters)  forecast  is  added  (LMMIM06,  1987 :Ch  7,  8).  As  a 
result,  the  last  quarter's  requirements  for  each  fiscal  year 
will  be  predicted  12  times  (once  every  quarter)  over  a  three 
year  period  before  the  quarter  is  entered  into.  The 
assumption  behind  Mr.  Novak's  analysis  was  that  the  first 
time  a  quarter  was  predicted,  it  represented  purely 
condemnations  spares  requirements.  None  of  the  changes 
associated  with  demand  volatility  would,  he  presumed,  be 


made  until  closer  to  the  actual  time  that  the  quarter 
arrived.  So  by  subtracting  the  first  prediction  for  a 
fiscal  quarter  from  the  twelfth  prediction  for  that  same 
quarter,  he  assumed  the  delta  would,  in  fact,  be  "demand 
volatility."  He  ran  this  analysis  for  several  aircraft  and 
received  several  different  churn  factors  (i.e.,  last 
estimate  divided  by  the  first  estimate).  His  bottom  line 
recommendation  was  to  support  a  2.5  churn  factor. 

By  his  own  admission,  the  number  of  aircraft  studied 
and  quarters  of  data  evaluated  were  insufficient  to  justify 
conclusive  findings.  The  Repairable  Stock  Program  Manager 
pointed  out  that  three  of  the  five  ALCs  feeding  quarterly 
inputs  into  the  D041  system  have  failed  to  update  the  system 
during  the  June  and  December  runs  for  as  long  as  he  could 
remember  (Rosenthal,  1991).  This  also  makes  the  results  of 
the  analysis  suspect. 

Mr.  Ray  Johnson,  a  Cost  Analyst  from  the  Air  Force  Cost 
Center,  performed  the  second  analysis,  dated  December  1988. 
He  used  weapon  system  obligation  data  from  the  Departmental 
On  Line  Accounting  and  Reporting  System  (DOLARS)  tracking 
located  at  the  Accounting  and  Finance  Center,  Lowry  APB,  in 
Denver,  Colorado  and  weapon  system  condemnations  costs  from 
the  Weapon  System  Cost  Retrieval  System  (WSCRS)  located  at 
Wright-Patterson  AFB,  Ohio  in  the  analysis.  In  his  study, 
Mr.  Johnson  compared  the  spares  obligation  data  for  twelve 
aircraft  from  FY  1979  to  FY  1985  with  the  condemnations 
spares  costs  associated  with  these  obligations.  He  dropped 
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three  of  the  resulting  ratios  out  of  the  analysis  because  he 
felt  they  were  outliers  and  averaged  the  remaining  nine 
values.  His  nine  aircraft  average  was  2.l5--the  same  value 
used  today  by  HQ  AFLC  as  the  standard  churn  value  for  all 
LSC  model  computations.  Mr.  Johnson,  lika  Mr.  Novak,  felt 
that  his  data  set  was  too  small  to  provide  conclusive 
results  (Johnson,  1991). 

To  perform  a  more  comprehensive  analysis  of  churn  for 
this  thesis,  another  approach  will  be  used  that  combines 
aspects  from  both  of  the  previous  studies.  An  ideal  data 
source  would  aggregate  spares  deliveries  by  weapon  system 
for  each  fiscal  year,  so  that  actual  deliveries  could  be 
compared  to  condemnations  to  develop  a  churn  factor.  The 
D041  system  receives  this  sort  of  delivery  data  at  the 
national  stock  number  level  but  it  doesn't  have  the 
capability  to  aggregate  the  information  to  the  weapon  system 
level  (as  needed  for  this  thesis).  Because  of  this,  spares 
deliveries  will  be  replaced  by  annual  replenishment  spares 
requirements  generated  by  the  D041  system  (CSIS)  and 
scrubbed  by  the  Recoverable  Stock  Program  Manager.  These 
requirements  will  be  compared  to  condemnations  to  develop  a 
churn  factor.  This  data  is  available  in  hard  copy  from  the 
Recoverable  Stock  Program  Manager  at  HQ  APLC/FMBSR. 
Condemnations  data  will  come  in  hard  copy  format  from  the 
WSCRS  system  at  HQ  AFLC/FMCA. 

Independent  Variable  Selection.  Physical  and 
performance  characteristics  of  aircraft  are  available  from  a 
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number  of  sources  (e.g.  Air  Force  Guide  No.  2,  Vol .  1 
(Green  Book)  and  2  (Brown  Book),  Rand,  Jane.' s  A1 1  the 
World's  Aircraft ) .  The  Green  Book  and  Brown  Book  contain 
official  Air  Force  "ble.ssed"  data  which  is  used  by  HQ 
AFLC/FMCA  in  their  anal/ses  and  these  sources  will  therefore 
be  used  (unless  specified  otherwise)  in  this  analysis. 

In  an  attempt  to  n;.rtow  down  the  candidates  for 
independent  variables,  ■>rior  parametric  models  were 
evaluated  to  see  which  ,-elationships  were  used  and  their 
supporting  logic  (see  subsection  entitled  "Previous  CER 
Work").  Additionally,  a  literature  search  was  conducted  for 
information  pertinent  to  the  independent  variable(s) 
selection.  The  balance  of  this  subsection  provides  the 
results  of  this  analysis,  beginning  first  with  casual 
relationships  which  were  not  included  in  this  thesis'  CERs 
and  then  providing  the  logic  behind  those  variables  which 
were  included. 

Component  reliabil  .ty  is  logically  related  to 
recoverable  spares  requ  .rements .  Less  reliable  parts  are 
more  likely  to  fail  and  require  more  maintenance  attention. 
If  the  failures  themselves  do  not  result  in  the  parts  being 
condemned,  the  increase<l  maintenance  activity  may  cause  wear 
and  tear  on  the  component  until  it  finally  must  be 
condemned.  Engineering  estimate  based  models  such  as  the 
Logistics  Support  Cost  (LSC)  model  use  factors  such  as 
component  mean  time  between  demand  (MTBD)  to  account  for 
reliability.  However,  no  equivalent  reliability  factor  at 
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the  weapon  system  level  of.  aggregation  was  identified  in  the 
literature  review. . 

A  RAND  study  discussed  several  complications  in 
determining  recoverable  spares  requirements.  Although 
engineering  estimate  based  models  typically  include 
reliability  factors  such  as  the  MTBD;  because  these  factors 
are  common  to  all  users  of  the  components,  they  lead  one  to 
assume  (erroneously)  that  all  users  should  expect  the  same 
levels  of  component  reliability.  However,  the  operating 
environments  in  which  the  components  are  placed  vary 
considerably  and  this  impacts  the  components’  reliability. 
For  example,  summer  temperatures  at  bases  like  Luke  AFB  are 
such  that  "temperatures  in  the  avionics  bay  of  idle  aircraft 
may  exceed  the  Mil  Specs  for  solid  state  devices"  (Crawford, 
1988:10).  At  night  the  t.^mperatures  at  Luke  AFB  cool  down 
significantly,  and  th*^  day  to  night  differential  is  cited 
for  causing  leaks  in  hydraulic  actuators  (Crawford, 

1988:10) . 

Changes  in  ma-’.ntenance  policies  and  training  programs 
will  impact  the  need  for  spare  parts.  Two  '’AC  initiatives-- 
Combat  Oriented  Supply  Organization  (COSO)  and  Combat 
Oriented  Maintenance  Organization  (COMO) --encouraged 
"increased  reliance  on  remove-and-replace  actions  instead  of 
remove-repair-and-replace  maintenance  actions"  (Crawford, 
1988:10).  Deployment  of  aircraft  to  Red  Flag  exercises  may 
be  accompanied  by  a  surge  in  spares  requirements  as  aging. 
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but  still  functioning  parts,  are  replaced  to  ensure  maximum 
performance  during  the  training  (Crawford,  1988:10). 

Finally,  the  RAND  study  revealed  that  "to  a  certain 
extent  demand  patterns  may  follow  parts  availability" 
(Crawford,  1988:10).  The  number  of  aircraft  continuing  to 
fly  with  a  certain  defective,  non-critical  part  may  continue 
to  rise  until  a  shipment  of  the  part  becomes  available--then 
a  surge  of  requests  for  the  part  come  in. 

All  of  the  complications  identified  in  the  RAND  study 
have  at  least  one  thing  in  common.  It  is  very  difficult,  if 
not  impossible,  to  capture  their  effects  in  generic  weapon 
system  CERs.  Even  detailed  engineering  estimates  do  not 
adequately  account  for  these  factors.  This  accentuates  the 
need  for  expert  judgement  in  the  requirements  development 
process.  "Good  models"  alone  will  not  guarantee  an  accurate 
estimate . 

The  number  of  levels  of  maintenance  (base, 
intermediate,  and  depot)  can  also  be  logically  tied  to 
recoverable  spares  requirements.  In  the  initial  spares 
arena,  as  the  number  of  repair  levels  increases,  so  does  the 
number  of  components  in  the  repair  "pipeline"  required  to 
achieve  support  objectives.  Levels  of  maintenance  can  also 
be  tied  to  replenishment  spares  due  to  the  fact  that 
pipeline  requirements  are  typically  paid  for  with 
replenishment  spares  funding  when  initial  spares  funds  are 
inadequate.  The  levels  of  maintenance  may  also  influence 
the  design  of  the  spares;  which,  in  turn,  may  affect  the 
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replenishment  spares  requirement.  The  relationship  of 
maintenance  levels  to  recoverable  spares  is  more  direct  in 
the  initial  spares  arena,  and  because  an  initial  spares  CER 
will  not  be  developed  in  this  thesis,  no  attempt  was  made  to 
determine  the  number  of  levels  of  maintenance  associated 
with  each  of  the  data  base  MDS. 

The  "bath  tub  curve"  theory  of  reliability,  as 
presented  in  AMGT  559,  "Life  Cycle  Cost  and  Reliability,"  at 
the  Air  Force  Institute  of  Technology  (AFIT),  was  already 
mentioned  in  defining  the  scope  of  this  thesis.  This  theory 
suggests  that  component  failure  rate  (and  therefore 
condemnations)  is  a  function  of  component  age--with  failure 
rates  that  are  higher  during  the  early  and  late  stages  of  a 
weapon  system's  life  and  relatively  constant  in  between 
these  extremes  (Gill,  1991:21).  The  "inverse  of  average 
fleet  age"  is  used  in  the  ALERT  spares  model  (Rexroad,  Lucas 
and  Collins,  1989:2)  and  the  "average  fleet  age"  is  used  in 
a  RAND  spares  model  (Crawford,  Landsdowne,  and  Finnegan, 
1908:22-23)  to  account  for  this  relationship.  However, 
given  that  the  scope  of  this  thesis  has  been  limited  to 
mature  weapon  systems  (where,  according  to  theory  the 
failure  rate  is  reasonably  constant),  this  type  of  factor 
should  have  no  impact  on  the  recoverable  spares 
requirements . 

An  unsuccessful  attempt  was  made  to  evaluate  the 
"inverse  of  average  fleet  age"  factor  as  a  check  on  the 
"mature  weapon  system"  constraint  placed  on  this  thesis's 
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data  base.  The  historical  database  for  this  variable  (used 
in  the  ALERT  model)  was  at  the  "MD"  level  of  aggregation 
instead  of  "MDS,"  No  other  source  for  this  variable  was 
identified.  If,  when  evaluated,  the  variable  had  been 
statistically  significant,  this  would  have  been  considered 
an  indication  that  either:  1)  immature  or  old  weapon  system 
data  was  erroneously  included  in  the  data  (i.e.,  the 
separation  of  weapon  systems  as  being  early,  mature,  or  old 
was  done  inaccurately),  or  2)  the  bath  tub  theory  is 
difficult  to  apply  on  real  weapon  systems  that  experience 
numerous  modifications  and  therefore  have  components  with  a 
myriad  number  of  different  ages. 

Not  only  is  it  logical  to  assume  that  components  will 
deteriorate  with  age,  it  is  also  logical  to  assume  that 
greater  utilization  of  components  will  cause  them  to  wear 
out  due  to  repeated  operational  stress.  One  spares  model 
accounted  for  this  using  annual  "flying  hours"  as  an 
independent  variable  (Crawford,  Lansdowne,  and  Finnegan, 
1988:24). 

Unless  one  knows  something  about  the  intensity  of  these 
flying  hours  (i.e.,  the  number  of  aircraft  the  hours  are 
spread  over),  however,  this  measure  does  not  logically  seem 
to  measure  physical  stress  as  much  as  it  measures 
"opportunities  for  something  to  go  wrong"  (e.g.,  bird 
strikes).  This  doesn't  invalidate  "flying  hours"  as  a 
potential  cost  driver.  It  simply  changes  its  underlying 
logic.  Its  effect  is  similar  in  nature  to  a  RAND  spares  CER 


cost  driver — the  annual  number  of  "active  aircraft"  in  the 
MDS  inventory  (Hoffmayer,  Finnegan,  Jr.,  and  Rogers, 
1980:16).  For  this  thesis,  "annual  flyiiig  hours",  "number 
of  aircraft"  and  "annual  flying  hours  divided  by  the  number 
aircraft"  will  all  be  evaluated. 

For  some  components  the  "number  of  sorties  per 
aircraft"  or  "landings  per  aircraft"  would  be  a  better 
indicator  of  physical  stress  than  "flying  hours  per 
aircraft"  (e.g.,  landing  gear).  The  anmial  numbers  of 
sorties  and  landings  were  obtained  from  AFALDP  800-4  (also 
referred  to  as  AFALCP  800-4  and  ALDP  800-4  in  later 
versions),  "Acquisition  Management  Aircraft  Historical 
Reliability  and  Maintainability  Data."  The  numbers  of 
"sorties"  and  "landings"  are,  once  again,  considered 
measures  of  opportunities  for  things  to  go  wrong.  The 
"annual  sorties  per  aircraft"  and  "annua  landings  per 
aircraft"  are  evaluated  as  measures  of  utilization 
intensity.  "Annual  flying  hours  divided  by  annual  sorties" 
was  evaluated  as  an  indicator  of  MDS  mission  profiles. 
Greater  sortie  lengths  should  imply  greater  utilization  of 
some  components  while  shorter  sortie  lengths  would  imply 
greater  utilization  of  others. 

It  should  be  noted  that,  due  to  holes  in  the  annual 
sorties  and  annual  landings  data,  those  models  developed 
with  these  variables  or  derivations  of  these  variables  had 
significantly  smaller  databases. 
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Some  components  are  not  being  utilized  during  flight 
and  others  are  utilized  even  before  flight  begins.  However, 
given  the  weapon  system  level  of  aggregation  in  this  thesis, 
it  was  not  possible  to  account  for  individual  component 
utilization  differences. 

Dummy  variables  will  be  used  to  separate  the  MDS  by  two 
classes  of  aircraft:  1)  f ighter/attack/trainer/f ighter- 
bomber  and  2)  bomber/cargo/tanker.  It  is  logical  to  assume 
that  relatively  light,  highly  maneuverable  aircraft 
(fighter/  attack/trainer/fighter-bomber)  put  different  kinds 
of  stresses  upon  their  components  than  heavy,  unmaneuverable 
aircraft  (bomber/cargo/tanker). 

Engine  spares  account  for  a  large  percentage  of  the 
total  recoverable  spares  requirement  (Steinlagy,  1991)  and 
therefore  it  is  appropriate  to  evaluate  independent 
variables  which  relate  to  this  weapon  system  subsystem's 
requirements.  "Thrust  per  engine"  is  cited  as  an  indicator 
of  engine  complexity  (Issacs,  Montanaro,  and  Olivo, 
1986:169).  As  this  variable  increases,  one  expects  the 
number  and  cost  of  engine  spares  required  to  increase.  For 
this  thesis,  maximum  sea  level  static  thrust  is  used.  The 
"number  of  engines"  per  weapon  system  is  related  to  the 
number  of  components  which  can  be  condemned  and  therefore  to 
the  spare  requirement  as  well. 

Another  factor  which  contributes  to  the  number  of 
components  is  the  weapon  system  size.  Two  types  of  sizing 
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factors  will  be  evaluated  for  this  thesis:  1)  ’’empty  weight" 
and  2)  aircraft  "length  plus  span." 

Finally,  weapon  system  technical  complexity  and 
performance  are  thought  to  influence  the  cost  of  annual 
spares  requirements.  Logically,  one  would  expect  high 
performance  aircraft  with  high  technical  complexity  to  place 
greater  stress  on  their  components  during  operations. 
Additionally,  the  individual  components  are  expected  to  be 
more  expensive  for  the  high  performance,  high  technical 
complexity  aircraft.  Relating  this  logic  to  another 
transportation  medium,  one  would  expect  a  greater  nu.mber  of 
failures  and  more  expensive  failures  on  an  Indy  race  car 
than  on  the  family  van. 

In  addition  to  the  "thrust  per  engine"  variable 
mentioned  earlier,  other  candidate  variables  examined  as 
performance  and  technical  complexity  drivers  include: 
"maxload  factor"  (i.e.,  how  many  "Gs"  was  the  aircraft 
designed  to  withstand),  the  "ratio  of  maximum  takeoff  weight 
to  empty  weight",  the  "maximum  rate  of  climb  at  sea  level," 
and  the  "maximum  combat  radius." 

Data  Population.  The  range  of  the  aircraft 
"population"  was  largely  determined  by  the  availability  of 
data  meeting  the  "mature  weapon  system"  criteria  established 
in  the  thesis  scope.  The  number  of  MDS  included  in  the 
independent  variable  database  was  smaller  for  some  of  the 
variables.  While  most  variables  had  five  years  of  data  for 
nineteen  MDS  (95  data  points),  "annual  landings"  data  was 
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only  available  for  sixteen  MDS  for  varying  numbers  of  years 
{ 41  data  points) . 

Parametric  Model  Technique 

As  discussed  before,  part  of  the  motivation  for  this 
thesis  comes  from  the  critical  reviews  being  given  to  the 
current  factor-based  approaches  used  to  develop  spares 
estimates.  This  dissatisfaction  was  expressed  in  a  SAP/FMC 
message,  dated  October  1990,  and  coordinated  on  by  SAF/AQK, 
SAF/AQX  and  USAF/LEX  (Robinson,  1991).  The  message  stated 
their  preference  that  demand  based  models  be  used  during 
milestone  reviews  and  for  independent  cost  estimates.  They 
questioned  the  use  of  factor  based  estimates  when  the  data 
required  for  demand  based  models  is  usually  available  by 
Milestone  II  (program  initiation  approval). 

The  loss  of  insight  into  underlying  causal 
relationships  in  the  factor  based  method  is  not  inherent  in 
all  parametric  models,  however.  The  problem  lies  in  the 
fact  that  costs  based  on  flyaway  cost  alone,  although  easy 
to  use,  do  not  include  the  driving  forces  behind  the 
estimate.  This  thesis  will  attempt  to  develop  a  CER(s)  with 
both  increased  visibility  of  its  underlying  causal 
relationships  and  the  ease  of  use  which  is  characteristic  of 
parametric  models. 

Ease  of  use  is  an  important  consideration  when  analysts 
lack  either  the  time  or  the  expertise  required  to  derive 
estimates  from  the  more  complicated  demand  based  approaches. 
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It  does  no  good  to  have  a  more  powerful  model  if  no  one  can 
use  it. 

Because  the  models  are  designed  for  use  early  in  the 
acquisition  life  cycle,  the  aggregate  level  for  the  model 
will  be  at  the  weapon  system  level  or  perhaps  at  the  major 
subsystem  level.  This  is  due  to  the  limited  data  input 
available  prior  to  completion  of  full  scale  development 
(TASC,  1989,  Ch  5,  55).  CERs  are  well  suited  for  dealing 
with  high  aggregate  level  relationships. 

Linear  regression  describes  relationships  between 
dependent  and  independent  variables  in  mathematical  terms 
and,  therefore,  is  the  logical  choice  for  CER  development. 

Regression  Assumptions 

The  following  assumptions  are  required  for  the  validity 
of  classical  regression  analysis  (Murphy,  1990-1991): 

1)  The  appropriate  cost  drivers  (independent  variables) 
are  included  in  the  model . 

2)  The  regression  model  specifies  the  correct 
relationship  between  the  dependent  and  independent 
variabl es . 

3)  The  independent  and  dependent  variable  data  come 
from  random  distributions. 

4)  There  is  no  bad  data  (i.e.  free  of  measurement 
error,  bias,  and  anomalies). 

5)  The  regression  error  terms  all  come  from  a 
normal  distribution  with  a  mean  of  zero. 

6)  The  error  terms  have  constant  variance  and  are 
independent  from  one  another. 

7)  The  cost  drivers  are  independent  from  one  another. 
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8)  Predictions  of  future  requirements  can  be  made  by 
examining  past  requirements. 

The  term  "assumption"  is  a  little  misleading  in  that  it 
implies  that  these  properties  are  taken  for  granted.  This 
is  only  the  case  when  one  is  evaluating  a  poorly  documented 
regression  model.  There  are  tests  which  can  be  performed  to 
provide  the  analyst  confidence  that  these  "assumptions"  are 
in  fact  true.  The  assumptions  and  the  corresponding 
diagnostic  tests  will  be  discussed  in  more  detail  in  the 
methodology  section  of  the  thesis. 

Previous  CER  Work 

Using  CERs  to  predict  spares  costs  is  not  a  new  idea. 
However,  the  fact  that  previous  parametric  models  have  been 
developed  does  not  negate  the  need  for  new  models.  Because 
parametric  models  are  based  on  historical  data,  they  should 
be  continuously  revised  and  updated  to  reflect  the  most 
current  available  data.  It  is  easier  to  validate  an 
existing  model  with  new  data  than  to  start  from  ground  zero. 
A  search  was  conducted,  therefore,  to  identify  existing 
spares  parametric  models.  The  following  models  will  be 
discussed  in  this  section:  1)  Rand  model,  2)  Modular  Life 
Cycle  Cost  Model  (MLCCM) ,  3)  Air  Logistics  Early 
Requirements  Technique  (ALERT)  model,  4)  Oversight  of 
Resources  and  Capability  for  Logistics  Effectiveness 
(ORACLE)  model,  and  5)  Levine  and  Horowitz  Study  [The  first 
three  models  meet  the  selection  criteria  for  inclusion  in 
Appendix  C  and  are  included  there  to  make  the  appendix  a 


stand  alone  document.  The  discussion  concerning  the  model's 
independent  variable  logic  is  more  extensive  in  this  section 
than  in  Appendix  C  because  of  its  relevance  to  the  selection 
of  cost  drivers  for  the  CER(s)  developed  in  this  thesis. 
Appendix  C  also  contains  evaluations  of  non-parametric 
models].  A  standard  format  will  be  followed  as  closely  as 
possible  for  ease  of  comparison  between  the  five  models. 

Mg.del.  #1 :  Rand  Model  .  All  references  in  this 
subsection,  unless  identified  otherwise,  come  from  the 
developer's  (see  below)  1980  report.  Estimating  USAF 
Aircraft  Recoverable  Soares  Investment . 

Developer(s) ;  K.J.  Hoffmayer,  F.W.  Finnegan,  Jr., 
and.W.H.  Rogers  of  the  Rand  Corporation,  August  1980. 

Model  Purpose :  The  model  is  an  update  to  a  1976 
Rand  model  for  estimating  USAF  aircraft  recoverable  spares 
investment.  It  includes  models  for  estimating  total 
replenishment  spares  requirements  at  the  major  subsystem 
level  (airframe,  avionics,  and  propulsion)  and  for 
estimating  condemnation  spares  requirements  at  the  same 
level  (v).  An  attempt  was  made  to  develop  an  initial  spares 
model  but  this  was  unsuccessful  due  to  limited  data 
availability  (23).  The  models  provide  annual  estimates  for 
peacetime  operating  stock.  War  readiness  material,  spare 
engines,  and  engine  spare  parts  are  excluded.  The  models 
are  intended  for  use  prior  to  the  preproduction  or 
deployment  decision  stages  of  the  acquisition  life  cycle 
(iii)  . 
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Model  Algorithm:  Table  1  provides  a  list  of  the 
independent  variables  associated  with  each  subsystem  CER. 

Table  1 


Subsystem  CER  Independent  Variables  (12-14) 


Airframe  CER 

Avionics  CER 

Propulsion  CER  | 

Total  active 

1  aircraft 

inventory  of  the 
given  MBS 

Total  active 
aircraft 

inventory  of  the 
given  MBS 

Total  number  of 
installed  engines 
in  the  MBS  force 

Airframe  flyaway 
cost 

Avionics  flyaway 
cost 

Propulsion  1 

flyaway  cost  1 

Peak  flying  hours 
per  MBS  per  year 

Bummy  variable  for 
bomber 

Bummy  variable  for 
reconnaissance 

Bummy  variable  for 
fighter/ attack 

Cargo  dummy  var. 

1 

Bummy  variable  for 
tanker 

1 

The  following  logarithmic  form  is  common  to  each  of  the 
subsystem  CERs  (11): 

log  =  log  a  log  (1) 

where 


y^f  =  investment  in  POS  spares  inventory  of  aircraft 
subsystem  i  at  time  t. 

=  the  jth  characteristic  observed  on  aircraft 
subsystem  i  at  time  t. 

a  and  Pj  =  regression  coefficients. 

=  the  error  for  aircraft  subsystem  i  at  time  t. 

The  errors  are  assumed  to  be  independent  across 
subsystems  but  correlated  over  time  with 
subsystem. 
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The  source  document  states  that  a  logarithmic  form  was 
chosen  to  develop  the  CERs  due  to  their  superior  handling  of 
heteroscedasticity  and  their  more  "real  world" 
multiplicative  nature  (11,  15).  They  fail  to  clearly  state 
whether  the  transformations  are  natural  (In)  or  common  (log) 


logarithmic  in  nature. 


Data  Inputs  and  Sources :  ^ost  data  from  1975  to 


1978  was  provided  for  the  following  iircraft  (7): 


A-7D 

B-52D 

B-52G 

B-52H 


C-5A 

KC-135A 

C-141A 

F-4C 


RF-4C 
F-4D 
F-4E 
F-111  \ 


F-lllD 

F-lllF 

T-37B 

T-38A 


Specific  data  elements  and  sources  ace  as  follows  (4-5): 


D041  "Recoverable  Consumption  Item  Requirements  System: 


National  Stock  Number  (NSN) 

Unit  Price  (in  then-year  dollars) 

Program  Begin  Date  (earliest  record  of  use) 

Program  Selection  Code  (Material  Program  managing  part) 
Organization  Field  Maintenance  (OFM)  Total  Demand 
Rate  (total  item  demand  expressed  in  terms 
appropriate  for  its  material  program) 

Base  Level  Condemnations  (NSN  level  condemnations  at 
base  level) 

Depot  Level  Condemnations  (NSN  level  condemnations  at 
depot  level) 

Total  Overhaul  Condemnations  (NSN  level  condemnations 
resulting  from  planned  overhauls) 

Total  Peacetime  Operating  Stock  Assets 
Application  (the  mission  design  series  (MDS)  or  other 
stock  number  using  the  item) 

Quantity  Per  Application 

J041  "Procurement  History  File:" 


National  Stock  Number  (NSN) 
Contract  Date 


Amount  of  Contract  ($) 

Quantity  Procured 

"Aerospace  Vehicle  Inventory  Status  and  Utilization  and 
Reporting  System"  (AVISURS): 
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Aircraft  MDS 

Calendar  Year  and  Month 

Flying  Hours 

Sorties 

Landings 

Average  Number  of  Possessed  Aircraft 


Other  data  used  in  the  model  development  was  obtained  from 
the  following  sources: 

*  TO  0025-30,  Technical  Manual,  "Unit  Cost  of 
Aircraft,  Guided  Missiles,  and  Engines." 

*  USAF  Statistical  Digests 

*  PA,  "USAF  Program,  Aerospace  Vehicles  and  Flying 
Hours." 

Assessment :  The  accuracy  of  the  models  provided 
are  difficult  to  assess  due  to  the  minimal  coverage  of  model 
diagnostics.  Because  Rand  does  not  specify  which  base  was 
used  in  the  logarithmic  transformations  (natural  or  common), 
the  reader  is  left  to  guess  at  the  significance  of  the 
standard  error  of  the  estimate  (the  SEE  is  a  measure  of 
prediction  accuracy  for  a  transformation  using  the  natural 
base) (Murphy ,  1990-1991).  The  propulsion  model  CER  has  poor 
statistics:  R^  of  .5841  and  SEE  of  .67318. 

It  is  unclear  if  the  logarithmic  form  is  really 
appropriate  because  they  never  actually  state  that  they 
observed  heteroscedasticity  in  the  data;  or  why  they  feel  a 
■multiplicative  equation  is  more  "real  world".  They  fail  to 
document  any  diagnostics  performed  or  other  models  attempted 
and  discarded. 

There  was  limited  discussion  concerning  the  logic 
underlying  the  independent  variables  chosen  for  the  CERs. 


35 


The  manner  in  which  subsystem  flyaway  costs  were  included  in 
each  of  the  subsystem  CERs  may  be  questioned.  One  tenet  of 
the  integrated  logistics  support  philosophy  is  that  as  an 
item's  reliability  improves,  the  reduction  in  its  operations 
and  support  costs  over  its  life  cycle  more  than  compensates 
for  its  increased  acquisition  cost  (which  has  resulted  from 
its  improved  reliability)  (TASC,  1989:Ch  5,  25).  The 
multiplicative  nature  of  the  subsystem  CERs  does  not  account 
for  this  belief  logically.  According  to  these  CERs, 
increased  acquisition  costs  will  always  result  in  greater 
spares  costs. 

Although  the  source  text  does  not  explain  its  rationale 
for  using  component  flyaway  costs,  a  case  can  be  made  that 
life  cycle  O&S  costs  reduce  with  improved  reliability  for 
reasons  other  than  reduced  spares  requirements.  After  all, 
replenishment  spares  costs  are  only  a  subset  of  the  total 
OSS  costs.  More  reliable  parts  should  fail  less  often  and 
therefore  cost  less  for  maintenance  and  repair.  If  RAND 
believes  that  these  types  of  savings  exceed  the  increased 
spares  costs  associated  with  more  expensive  components,  then 
their  model  logic  is  not  contradictory  to  reliability 
theory. 

In  addition  to  airframe  flyaway  cost,  the  airframe  CER 
included  several  additional  cost  drivers:  1)  total  active 
aircraft  inventory  of  the  MDS,  2)  peak  flying  hours,  and  3) 
mean  organization  field  maintenance  total  demand  rate. 

Total  active  aircraft  inventory  is  a  logical  cost  driver. 
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More  spares  are  required  as  the  number  of  aircraft 
increases . 

Peak  flying  hours  was  used  in  lieu  of  programmed  flying 
hours  because  it  gave  "a  better  statistical  fit"  (16).  The 
authors  stated  that  this  resulted  from  "significant 
inventories  of  stock"  remaining  which  had  been  purchased  to 
support  the  Vietnam  war  (data  is  from  1975  to  1978)  (16). 
Although  this  distinction  was  not  clear  to  the  author,  the 
use  of  some  sort  of  flying  hour  program  is  logical  because 
one  would  expect  that  the  more  wear  and  tear  that  is  placed 
on  an  aircraft,  the  greater  the  number  of  condemnations. 

The  mean  organization  field  maintenance  total  demand 
rate  (OFMTDR)  was  included  as  a  "measure  of  airframe 
reliability"  (17).  Component  reliability  can  be  logically 
tied  to  condemnations  because  unreliable  parts  will  fail 
more  often  and  may,  in  the  process,  be  damaged  beyond  repair 
or  cause  irreparable  damage  to  other  components. 
Additionally,  even  parts-,  that  can  be  easily  repaired  may 
receive  wear  and  tear  ii.  the  maintenance  process  itself 
(e.g.,  stripped  bolts,  etc)  and  therefore  are  condemned 
sooner  than  more  reliable  parts.  In  this  model,  however, 
the  manner  in  which  the  OFMTDR  was  obtained  is  not 
consistent  with  application  of  the  CER  in  early  stages  of 
aircraft  development  (w  lere  the  authors  claim  their  CERs  are 
applicable).  This  variable  was  a  "mean,  weighted  by  the 
total  item  count,  of  th«  OFMTDRs  of  all  the  recoverable 
airframe  items"  (17).  .since  a  complete  list  of  airframe 


items  is  unlikely  to  be  available  during  early  stages  of 
aircraft  development,  it  is  unclear  hov?  this  type  of 
statistic  will  be  readily  available. 

The  Avionics  CER  included,  in  addition  to  avionics 
flyaway  costs,  dummy  variables  to  distinguish  the  mission  of 
the  aircraft  (bomber,  reconnaissance,  fighter/attack,  cargo, 
or  tanker)  and,  once  again,  the  total  number  of  active 
aircraft  in  the  MDS  inventory.  It  is  logical  that  the 
amount  of  aircraft  avionics  equipment  will  vary  by  mission 
type,  and  the  number  of  spares  required  should  increase  as 
the  amount  of  avionics  equipment  increases.  Total  aircraft 
inventory  is  logical  for  the  same  reason  explained 
previously. 

The  propulsion  CER  included  only  one  additional  cost 
driver  besides  the  propulsion  flyaway  costs.  The  total 
number  of  installed  engines  in  the  MDS  force  is  a  logical 
cost  driver — the  number  of  spares  required  should  increase 
as  the  number  of  engines  increases.  It  should  be  noted  that 
this  CER  includes  no  cost  drivers  which  will  vary  over  time. 
Engines  spare  parts  account  for  the  majority  of  the 
condemnation  spares  requirements  (as  seen  in  the  dependent 
variable  data  set)  and  yet  this  CER  would  lead  one  to 
believe  that  propulsion  spares  remain  relatively  constant 
over  time.  This  is  in  contradiction  to  the  "bathtub"  theory 
of  reliability  which  states  that  failures  (and  logically 
therefore  condemnations)  are  greater  early  in  a  weapon 
system's  life  due  to  manufacturing  defects  and  later  in  life 
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due  to  wear  out  (Crawford,  Lansdowne  and  Finnegan,  1988:23; 
Gill  :21) . 

The  RAND  model  was  evaluated  by  an  Initial  Spares 
Working  Group  comprised  of  twenty  two  members  representing 
HQ  APLC,  ASD  and  HQ  AFSC,  They  concluded  that  the  model's 
database  should  be  updated  and  a  clear  distinction  made 
between  initial  and  replenishment  spares  before  they  could 
use  the  model  (Rexroad  and  others,  1990:11). 

Model  #2 :  Modul ar  Life  Cycle  Cost  Model  (MLCCM) .  A1 1 
references  in  this  subsection,  unless  specified  otherwise, 
come  from  Grumman  Aerosoace  Corporation's  1986  report 
entitled  Modular  Life  C vcle  Cost  Model  for  Advanced  Aircraft 
Systems ,  Cost  Methodology  Development  and  Application.  The 
authors  were  R.  Isaacs,  N.  Montanaro,  and  P.  Olivo. 

Developer(s) ;  Grumman  Corporation,  Program  Team 
directed  by  Mr.  R.  Isaacs,  September  1986. 

Model  Purpose :  The  MLCCM  is  a  parametric-based 
series  of  models  for 


predicting  advance!  technology  aircraft  costs,  to  the 
major  subsystem  le/els,  for  the  Research,  Development, 
Test,  and  Evaluation,  Production,  Initial  Support,  and 
Operations  and  Supoort  phases  of  the  system  life  cycle 
during  conceptual  and  preliminary  design.  (Isaacs, 
1986:iii) 


Initial  and  replenishment  spares  are  but  subsets  of  the 
overall  costs  within  the  production  and  O&S  periods, 
respectively.  Their  cost  is  broken  out  for  14  subsystems 
(structure,  crew  system,  landing  gear,  flight  control,  cargo 
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handling,  engines,  engine  installation,  environmental 
control  systems,  electrical,  hydraulic/pneumatic,  fuel 
system,  avionics,  armament,  auxiliary  power  unit)  (11)  for 
two  classes  of  aircraft  ( fighter/ attack/bomber  and 
cargo/transport/tanker) (x) . 

Model  algorithm:  Because  the  MLCCM  was  developed 
as  a  tool  for  conducting  trade  studies  during  the  design 
stage,  the  CERs  were  developed  using  a  Work  Breakdown 
Structure  (WBS)  format.  In  this  way  the  design  engineers 
would  be  able  to  relate  costs  to  the  WBS  elements  for  which 
they  were  responsible.  Step-wise  regression  was  used  to 
develop  log-linear  regression  equations.  ."Igain,  it  is 
unclear  if  they  used  natural  or  common  base  transformations. 
They  limited,  for  most  cases,  the  number  of  parameters  in 
any  CER  to  one  third  the  number  of  data  points  (47-48). 

While  it  makes  sense  to  preserve  degrees  of  freedom  by 
limiting  the  number  of  variables  used  (compared  to  the 
number  of  data  points),  it  isn't  clear  why  the  developers' 
chose  one  third  as  a  criteria  ratio. 

Data  Inputs  and  Sources :  Cost  data  was  derived 
from  several  sources  including:  "Visibility  and  Management 
of  Operating  Support  Costs"  (VAMOSC)  system,  APR  173-13 
Factors,  and  the  1975/1976  Operating  and  Support  Cost 
Estimating  Report .  Independent  variable  technical  data 
sources  include:  Standard  Aircraft  Characteristics  (SAC) 
charts,  group  weight  statements,  technical  orders,  and  the 
manufacturer . 
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The  SAC  charts  were  used  to  obtain  data  on  engine 
design  and  performf.nce,  fuel  and  tankage,  armament, 
loading  and  aircraft  performance,  development  dates, 
etc.  Weights,  areas,  volumes,  dimensions,  and  general 
aircraft  design  characteristics  were  obtained  from  the 
group  weight  statements  and  the  manufacturers.  Plight 
manuals  were  used  for  data  on  electrical,  flv  ,  ’  power, 
and  flight  actuator  systems  and  for  general  a  ‘rcraft 
design  characteristics  as  well  [note  that  not  all  of 
these  variables  are  used  in  the  replenishment  spares 
CERs].  (13) 


Assessment ;  The  fact  that  the  models  were 
developed  with  obligation  data  brings  with  it  all  the 
uncertainty  previously  discussed  in  the  data  problem 
section.  The  report  admits  the  need  for  better  cost  data 
inputs  (219). 

The  model  statistics  provided  were  not  complete,  "R" 
values  were  provided  as  opposed  to  the  "R^"  statistic 
commonly  seen.  The  R^  values  for  5  of  the  14  subsystems  in 
the  fighter/attack/bomber  class  of  aircraft  were  poor  (less 
than  .7)  (165-169).  No  discussion  of  model  diagnostics  was 
provided. 

The  independent  variables  used  in  the  subsystem  CERs 
differ  between  two  general  classes--those  for  cargo/trans¬ 
port/tanker  aircraft  and  those  for  fighter/attack/bomber 
aircraft.  The  following  text  describes  the  independent 
variables  used  in  the  cargo/transport/tanker  class  and  how 
the  variables  logically  impact  subsystem  spares 
requirements : 

1)  Structure  CER:  "Cargo  weight"  and  "cargo 
volume"  were  included  as  measurements  of  aircraft  size. 
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"The  size  of  the  aircraft  directly  relates  to  the  amount  of 
structural  spares  required"  (209). 

2)  Crew  Systems  CER:  The  "number  of  primary 
compartments"  and  the  "length  +  span"  (LENSPN)  were  included 
as  measures  of  the  fuselage  interior.  "As  such  [they] 
reflect  a  degree  of  the  demands  imposed  on  the  crew  system. 
Increased  size  results  in  larger  crews,  seats,  etc,  and  as 
such  results  in  an  increase  in  replenishment  spares  costs" 
(209).  Apparently  the  logic  here  is  that  larger  crews 
require  bigger  "crew  systems,"  composed  of  more  parts,  and 
therefore  more  spare  parts  are  needed. 

3)  Landing  Gear  CER:  The  "number  of  landing  gear 
wheels"  is  included  as  "indicative  of  the  number  of  wheels, 
tires,  brakes,  and  associated  hardware  requiring  repair 
action  and  maintenance  attention"  (210).  "An  increase  in 
the  number  of  wheels  results  in  an  increase  in  [subsystem] 
replenishment  spares  costs"  (210). 

Additionally,  the  "Mass  times  Velocity  Squared"  (MVSQ) 
is  included  as  a  measure  of  the  energy  absorption 
requirement  placed  on  the  landing  gear.  "An  increase  in  the 
MVSQ  results  in  abuse  to  the  wheels,  tires  and  brakes. 
Consequently,  an  increase  in  replenishment  spares  costs  will 
result"  (210). 

4)  Flight  Controls  CEF::  The  "number  of  flight 
control  actuators"  (NOACTS)  is  included  as  "an  indicator  of 
the  overall  complexity  of  the  flight  control  subsystem  .  .  . 
[and]  one  of  the  largest  contributors  to  the  cost  of  spares" 
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(210) .  This  increased  complexity  is  said  to  cause 
"increased  removal  activity  of  the  major  flight  control 
components"  and  thus  an  increased  spares  requirement  (210). 
It  isn't  clear  if  this  means  complex  systems  have  a  greater 
number  of  parts  which  can  fail  or  if  it  takes  more  work  to 
remove  the  components  causing  more  wear  and  tear  that  is 
maintenance  related. 

The  "takeoff  gross  weight  maximum"  (TGWMAX)  is  included 
as  "a  measurement  of  aircraft  size  and,  as  such,  is  an 
indicator  of  flight  control  size.  The  larger  aircraft 
requires  more  flight  control  components  and,  as  such,  more 
spares"  (210). 

5)  Engine  Ins’.allation  CER:  Again,  TGWMAX  is 
included  as  a  measure  o:  aircraft  size.  Size  is  said  to  be 
proportional  to  spares  lost.  The  "engine  pressure  ratio  is 
a  measure  of  engine  sopliistication  and,  as  such,  has  an 
influence  on  the  engine  installation  component  requirements" 

(211)  [The  source  text  describing  the  subsystem  CERs  and 
their  supporting  logic  did  not  specify  what  type  of  "engine 
pressure  ratio"  is  used] .  An  increase  in  the  number  of 
installation  components  means  that  more  parts  are  available 
to  fail  and  therefore  more  spares  should  be  required. 

6)  Environmental  control  System  (ECS)  CER:  "Hours 
per  mission"  is  included  as  a  measure  of  aircraft  and 
component  usage.  "Maintenance  cost  increases  with  aircraft 
utilization"  (211). 
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"Fuselage  Volume"  (FLGVOL)  is  said  to  measure  the  "ECS 
size  and  complexity  relative  to  the  amount  of  cooling 
capacity"  (211).  As  this  variable  increases,  so  does  the 
size  and  number  of  ECS  components  resulting  in  increased 
spares  requirements. 

7)  Electrical  CER:  The  "number  of  generators"  is 
included  as  a  measure  of  the  "size  and  number  of  components 
in  the  electrical  generating  and  distribution  system"  (212) 
Again  increased  size  equates  to  increased  spares 
requirements . 

The  "cargo  floor  area"  is  a  measure  of  the  space 
requiring  electricity.  "An  increase  in  the  useable  floor 
space  results  in  an  increase  in  size  of  the  electrical 
subsystem  and  consequently  replenishment  spares  costs" 

(212). 

8)  Hydraulic/Pneumatic  CER:  The  LENSPN  and  "number 
of  hydraulic  pumps"  are  included  as  "measures  of  the  size 
and  complexity  of  the  aircraft  hydraulic  system  and 
indicative  of  the  number  of  supply  circuits  and  components 
required"  (212).  Again,  as  size  and  complexity  grows  the 
spare  requirement  is  also  said  to  grow. 

9)  Fuel  System  CER:  The  LENSPN  and  "fuel  system 
weight"  are  included  as  "measures  of  aircraft  size,  and 
hence,  fuel  system  size"  (213).  Increased  size  relates  to 
increased  spares  requirements. 

10)  Cargo  Handling  CER:  "Cargo  weight"  and  "cargo 
volume"  (CARVOL)  are  included  as  measures  of  the  cargo 
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handling  subsystem's  load  capacity.  As  the  number  and  size 
of  cargo  handling  equipment  increases,  so  does  the  number  of 
spares  required. 

11)  Auxiliary  Power  Unit  CER:  FLGVOL  and  CARVOL 
"are  sizing  factors  relative  to  the  amount  of  air 
conditioning  to  be  supplied  by  the  APU"  (213).  As  these 
variable  increase,  "larger  and  more  complex"  APUs  must  be 
used  and  thus,  an  increased  requirement  for  spares  (213). 

12)  Avionics  CER:  "Total  KVA"  and  "avionics  black 
box  weight"  were  included  as  measures  of  the  avionics 
subsystem  size  and  complexity.  Increases  in  these  variables 
are  said  to  result  in  increased  spares  costs. 

13)  Engine  CER:  "Sea  level  maximum  mach"  and 
"thrust  per  engine"  were  considered  measures  of  engine 
complexity.  "The  more  complex  the  engine,  the  more  costly 
are  the  spares,  therefore,  an  increase  in  these  parameters 
will  result  in  an  increase  in  the  cost  of  spares"  (169). 

This  CER  was  also  used  for  the  fighter/attack/bomber  class 
of  aircraft's  engine  CER. 

Rather  than  detailing  each  subsystem's  independent 
variables  for  this  second  class  of  aircraft,  the  following 
text  describes  only  those  cost  drivers  which  weren't  already 
mentioned  above. 

The  landing  gear  CER  for  this  class  included  "takeoff 
gross  weight  clean"  as  a  cost  driver.  This  was  said  to  be  a 
measure  of  the  "kinetic  energy  absorption  capability  and 
size  of  the  landing  gear  components,"  and  because  of  this. 
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it  directly  related  to  tire,  brake  and  wheel  wear,  and  thus 
the  need  for  more  spares. 

The  engine  installation  CER  included  "engine  thrust  to 
weight  ratio"  as  "a  degree  of  sophistication  and  size  [of 
what  the  documentation  does  not  say]  in  relation  to  the 
aircraft  performance  capabilities"  (166).  Increases  in 
these  factors  are  said  to  lead  to  increased  maintenance 
activity  and  thus,  the  need  for  more  spare.s. 

The  environmental  control  system  CER  includes  "BTU  per 
hour"  as  a  "measure  of  the  cooling  capacity  and  the  size  of 
the  ECS"  (167).  Increased  size,  once  again,  equates  to  a 
need  for  more  spares. 

"Avionics  installation  weight"  is  used  in  both  the 
electrical  CER  and  the  Avionics/Armament  CER.  This  weight 
is  said  to  be  a  measure  of  the  size  and  number  of  components 
in  the  avionics  subsystem.  An  increase  in  this  variable 
leads  to  increased  spares  requirements  for  both  subsystems. 

The  fuel  system  CER  uses  "internal  fuel  weight"  as  a 
measure  of  the  size  and  number  of  fuel  system  components. 

The  greater  the  number  of  parts,  the  greater  the  number  of 
spares  required.  Additionally,  "maximum  mach"  is  used  as  "a 
degree  of  sophistication  required  of  the  fuel  system"  (168). 
Increased  sophistication  is  said  to  require  more  parts  and 
thus  more  spares. 

The  discussion  of  independent  variable  selection 
contained  in  this  model's  documentation  needs  additional 
work.  In  many  cases  it  isn't  clear  why  one  measure  of  size 
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is  used  in  one  CER  and  another  size  measure  is  more 
appropriate  for  another  CER.  It  some  cases  it  appears  as 
though  the  authors  had  to  stretch  their  imagination  to  come 
up  with  the  logic  behind  their  cost  drivers.  This  might  be 
attributable  to  the  fact  that  step-wise  regression  was  used 
to  identify  the  cost  drivers.  The  logic  might  have  been 
developed  after  the  selection  of  cost  drivers. 

Model  #3;  Air  Logistics  Early  Requirements  Technique 
( ALERT ) .  Unless  stated  otherwise,  references  in  this 
subsection  will  come  from  a  1989  report  entitled  Air 
Logistics  Early  Requirements  Technique  (ALERT)  FY90-94 
Program  Objective  Memorandum  ( POM)  Forecast .  The  authors 
were  Adrienne  Rexroad,  Robert  Lucas,  and  Larry  Collins. 

Developer( s) ;  AFLC/MMM,  1984. 

Model  Purpose:  ALERT  has  been  used  since  1984  by 
HQ  AFLC  as  the  starting  point  for  developing  BP15  aircraft 
peacetime  spares  Program  Objective  Memorandum  (POM)  inputs. 
The  POM  is  the  Air  Force’s  long  range  budget  requirements 
document.  ALERT  is  the  "starting  point"  because  the  output 
from  ALERT  is  scrubbed  by  the  BP15  Program  Manager  prior  to 
its  submittal  (1). 

Model  Algorithm:  Sixteen  separate  CERs  are 
developed  with  straight  linear  regression  to  predict  the 
first  year  of  POM  requirements  for  thirteen  different  weapon 
systems,  the  F-lOO  engine,  common  spares  (to  multiple  weapon 
systems),  and  an  "other"  category.  This  first  year's 
estimate  is  then  used  as  historical  input  for  the  next  four 
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years’  predictions — a  regression  technique  which  is  referred 
to  as  "bootstrapping."  (2). 

Data  Inputs  and  Sources ;  The  dependent  variable 
data  used  were  the  requirements  submitted  in  the  last  Budget 
Estimate  Submission  (BES) .  There  were  a  total  of  four 
independent  variables  used  (not  all  at  once)  in  the  CERs:  1) 
Mission  Design  Series  buy  requirements  from  the  D041 
"Recoverable  Consumption  Item  Requirements  System,"  2) 
Average  Fleet  Value,  as  calculated  by  OSAF/AC,  3)  the 
reciprocal  of  the  estimated  Present  Fleet  Age,  also  provided 
by  USAF/AC,  and  4)  Chronological  Year. 

Assessment ;  Only  six  of  the  sixteen  weapon  system 
class  CERs  had  adjusted  R^  values  exceeding  .7  and  only  one 
class  exceeded  .75.  The  BP  Manager  scrub  that  followed 
subsequent  to  the  ALERT  run  changed  the  input  values 
further.  The  BP  Manager  is  critical  of  the  D041  input  data 
since  it  uses  the  June  data  run  (a  quarter  not  updated  by 
three  of  the  five  ALCs) .  He  also  questioned  the  logic  of 
using  the  fleet  value  as  a  cost  driver.  USAF/AC  based  their 
estimate  of  fleet  value  on  projected  future  flying  hour 
programs  which  decrease  over  time.  The  fleet  values, 
therefore,  decrease  over  time.  The  spares  requirement, 
however,  logically  gets  larger  as  the  fleet  gets  older  (7). 

A  1988  validation  study  of  the  ALERT  model  compared  the 
scrubbed  ALERT  forecast  to  three  other  forecasting 
approaches:  1)  cost  per  flying  hour  factors,  2)  inflation 
growth,  and  3)  unscrubbed,  "pure"  ALERT  forecasts.  This 
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analysts  concluded  that  the  scrubbed  ALERT  forecast  was  the 
most  accurate  approach  and,  in  fact,  this  technique 
predicted  (in  1984)  1987  spares  requirements  within  two 
percent  of  the  actual  obligations  recorded  that  year 
(Rexroad  and  Collins,  1938:8).  However,  it  should  be  noted 
that  this  "two  percent"  accuracy  was  for  the  total 
replenishment  spares  forecast.  Within  the  individual  CERs, 
several  of  the  estimates  missed  the  mark  considerably.  The 
B-52  estimate  ($148.2  million)  was  $109.2  million  higher 
than  the  funds  obligated.  The  F-16  estimate  ($372.2 
million)  was  $225.6  million  higher  than  actuals  and  the  F- 
100  engine  estimate  ($393.4  million)  was  $232.4  million 
lower  than  the  actual  obligations  (Rexroad  and  Collins, 
1988:7-8).  The  remarkable  "accuracy"  of  this  model, 
therefore,  could  be  nothing  more  than  the  Air  Force 
obligating  everything  they  had  to  obligate  (provided  as  a 
result  of  the  ALERT  forecast),  without  saying  anything  about 
whether  real  requirements  were  met. 

The  validation  study  pointed  out  several  concerns  with 
the  model.  A  chi-square  test  was  performed  on  all  four 
estimating  techniques  being  evaluated  and  it  was  determined 
that  "none  of  the  forecasting  approaches  are  statistically 
close  to  the  actual  obligated  dollars"  (Rexroad  and  Collins, 
1988:9).  Additionally,  the  authors  concluded  that  although 
a  high  correlation  existed  between  the  dependent  variable 
and  the  independent  variables  in  their  data  set,  they  found 
no  causality  and  therefore  had  no  "complete  'intuitive' 
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interpretation  of  why  past  relationships  exist"  (Rexroad  and 
Collins,  1988:5).  The  authors  recommended  that  additional 
analysis  be  performed  to  "include  variables  like  actual 
flying  hours,  type  of  mission,  and  the  expected  occurrence 
of  a  major  modification  to  predict  cause  and  effect 
relationships"  (Rexroad  and  Collins,  1988:10). 

A  1988  RAND  report  (see  next  model  below)  provided  a 
brief  evaluation  of  the  ALERT  methodology.  One  additional 
shortcoming  which  it  identified  is  that  the  model  does  not 
"correct  for  inflation  by  converting  dollar  estimates  into 
constant  dollars"  (Crawford,  Lansdowne,  and  Finnegan, 
1988:8). 

Model  #4:  Oversight  of  Resources  and  Capability  for 
Logistics  Effectiveness  (ORACLE) .  Unless  stated  otherwise, 
all  references  in  this  subsection  come  from  a  1988  RAND 
report  entitled,  ORACLE  and  Reguirements  Forecasting .  Vol . 

II :  Predicting  the  Peacetime  Spares  Reguirement .  The 
authors  are  Gordon  B.  Crawford,  Z.F.  Lansdowne,  and  F.W. 
Finnegan. 

Developer(s) :  RAND  Corporation  (see  authors 

above) . 

Model  Purpose:  ORACLE  is  "a  methodology  developed 
to  relate  dollars  expended  on  recoverable  components  to  the 
goals  set  in  the  Planning,  Programming,  and  Budgeting  (PPB) 
process"  (1).  The  ORACLE  methodology  was  developed  with 
three  hypotheses  in  mind: 
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1.  A  constant  requirement  methodology  and  the 
converting  of  prices  to  constant  year  dollars  with 
reasonable  inflation  indices  would  make  the  BP15 
requirement  more  stable  and  more  readily  predictable. 

2.  Breaking  the  total  requirement  for  a  weapon  system 
into  several  federal  stock  class  (FSC)  groupings  and 
analyzing  the  regression  of  each  group  on  the  several 
explanatory  variables  would  permit  identification  of 
certain  groups  that  do  not  regress  well  and  hence 
deserve  expert  attention  and  judgement  to  predict  their 
requirement . 

3.  Removing  these  'hard  to  predict*  groupings  for 
individual  attention  would  then  make  the  remainder  of 
the  expenditure  substantially  more  stable  and  easier  to 
predict.  (2) 


Model  Algorithm:  CERs  were  developed  for  three 
weapon  systems  (F-15,  F-16,  and  C-5).  Ten  years  of  data 
(1975  -  1984)  was  used  for  the  C-5.  In  the  case  of  the  F-15 
and  F-16,  the  data  bases  were  reduced  to  seven  years  (1978  - 
1984)  and  six  years  (1979  -1984)  respectively.  The 
modelers  felt  that  these  aircraft  were  too  new  to  the 
inventory  in  the  mid“70s  and  that  this  early  data  would  not 
accurately  predict  the  requirements  for  mature  weapon 
systems . 

The  modelers  performed  what  they  called  "a  loose 
approximation  of  what  others  might  call  a  regression 
analysis"  (19).  They  say  this  because  they  did  not  believe 
that  the  residuals  in  their  analysis  would  be  "normally 
distributed  with  a  precisely  described  covariance  matrix" 
(19)  [see  regression  assumptions  5  and  6  on  page  20]  and 
they  took  "repeated  and  excessive  liberties"  in  such 
subjective  areas  as  "rejecting  outliers  and  replacing  their 
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values"  (22).  The  modelers  claim  that  if  the  CERs  did  not 
perform  well  under  these  relaxed  conditions,  they  would 
perform  even  poorer  using  the  stricter  linear  regression 
assumptions  (20). 

The  dependent  variable  being  studied  for  each  weapon 
system  was  the  marginal  annual  cost  of  the  replenishment 
spares  requirement.  Given  the  small  size  of  the  P-16  data 
base,  the  authors  limited  the  number  of  coefficients  in  any 
one  CER  to  two  (assumably  to  preserve  degrees  of  freedom). 
The  independent  variables  evaluated  included  the  average  age 
of  the  fleet,  flying  hours  and  the  value  of  the  fleet. 

Little  explanation  is  given  for  why  "value  of  the  fleet"  is 
used  other  than  the  Navy  has  "long  used  it  to  predict  Naval 
air  requirements"  (22).  The  "average  age  of  the  fleet"  is 
included  to  account  for  the  "bath  tub"  theory  of  reliability 
(described  earlier  in  the  RAND  Model  evaluation)  (23).  The 
"Flying  hours"  variable  was  included  to  account  for  the  fact 
that  "many  spare  parts  fail  as  a  result  of  repeated  physical 
stress  or  wear  that  is  a  direct  result  of  flying,  taking 
off,  or  landing"  (24).  "Flying  hours"  were  used  in  lieu  of 
"aircraft  sorties"  due  to  better  availability  of  flying  hour 
data  and  the  modelers'  contention  that,  when  plotted,  a 
curve  describing  sorties  per  year  "tends  to  look  like  curves 
that  describe  flying  hours  per  year"  (24). 

As  stated  earlier,  the  modelers  wanted  to  determine  if, 
by  separating  hard  to  predict  PSC  groupings,  they  could 
improve  their  ability  to  predict  requirements  for  the 
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balance  of  the  weapon  system's  requirements.  In  other 
words,  they  wanted  to  separate  the  "art"  (hard  to  predict 
groupings  requiring  expert  judgement)  from  the  "science" 
(remaining  groupings)  in  spares  requirements  determination 
(25).  To  accomplish  this,  separate  CERs  were  developed  for 
fourteen  separate  FSC  classes  for  each  weapon  system,  in 
addition  to  the  overall  weapon  system  CER.  Those  FSC 
classes  which  failed  to  show  a  strong  statistical 
relationship  to  the  independent  variables  were  then  removed 
from  the  overall  weapon  system  data  base  end  the  impact  on 
its  CER  was  noted. 

Data  Inputs  and  Sources ;  The  dependent  and 
independent  variable  data  used  in  this  analysis  (discussed 
above)  was  taken  from  the  D041,  "Recoverable  Consumption 
Item  Requirements  System." 

Assessment ;  The  idea  of  separating  "hard  to 
predict"  components  from  the  total  weapon  system  data  base 
is  an  interesting  and  logical  idea.  According  to  the 
modelers'  conclusions,  they  did  find  support  for  their 
second  hypothesis  that  this  sort  of  separation  could  be 
accomplished.  Some  of  the  FSC  groupings  exhibited  "wild 
swings,"  and  the  modelers  felt  that  they  would  be  difficult 
to  predict  no  matter  what  independent  variables  were  used 
(41).  Unfortunately,  the  exclusion  of  these  FSC  groupings, 
according  the  modelers,  showed  no  marked  improvement  in  the 
ability  to  predict  requirements  for  the  remainder  of  the  FSC 
groupings.  The  first  hypothesis  (concerning  anticipated 
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improvements  garnered  from  a  constant  requirements 
methodology  and  constant  year  dollars)  was  also  not 
supported.  Additional  research  would  have  to  be  conducted 
to  determine  if  a  larger  data  base  or  different  independent 
variables  would  impact  these  results. 

Model  #5 :  Levine  and  Horowitz  Study.  Unless  stated 
otherwise,  references  in  this  subsection  will  come  from  a 
1989  study  conducted  by  Daniel  B.  Levine  and  Stanley  A. 
Horowitz  ent  tied,  Predicting  the  Cost  of  Initial  Spares . 

Model  Purpose:  The  purpose  of  this  study  was  to 
develop  a  CER  for  predicting  initial  spares  costs.  The  CER 
would  benefit  service  and  OSD  budget  planners  in  laying  in 
long  range  budgets  for  new  aircraft  many  years  before  their 
deployment  (1). 

Model  Algorithm;  The  authors  used  linear 
regression  to  test  hypothesized  relationships  in  data  from 
twenty  one  Navy  and  Air  Force  aircraft.  The  dependent 
variable  was  the  total  obligation  authority  (TOA)  for 
procurement  of  initial  spares  during  the  life  of  the  weapon 
systems'  programs.  Actual  obligation  data  is  used  from  1972 
-  1988,  and  TOA  is  included  through  1994.  Aircraft  mission 
design  series  (MDS)  selected  were  limited  to  those  MDS  for 
which  data  was  available  for  the  entire  program  (i.e.,  the 
program  started  after  1972  and  will  finish  before  1994). 
Independent  variables  evaluated  included  weapon  system  cost, 
empty  weight,  maximum  speed,  total  program  procurement 
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quantity,  and  duimy  variables  to  control  for  the  aircraft 
mission  (2). 

Three  criteria  were  established  for  selection  of  the 
final  CER^,s):  ^'jcit'''e  signs  for  the  cost  driver 

coefficients,  2)  high  and  3)  high  t-statistics  (2). 

This  study  resulted  in  the  development  of  four  CERs  which 
met  the  criteria. 

Data  Inputs  and  Sources ;  The  dependent  variable 
data  came  from  the  FY  1989  historical  procurement  annex 
covering  the  fiscal  years  1972  -  1994.  The  independent 
variable  data  (discussed  above)  came  from  "Standard  Aircraft 
Missile  Characteristics"  (Air  Force  Guide  No.  2)  and  Jane’s 
Al 1  The  World's  Aircraft  ( 5 ) . 

Assessment ;  The  use  of  actual  obligations  as  the 
dependent  variable  carries  with  it  all  the  uncertainties 
discussed  previously  in  the  subsection  entitled  "data 
problems"  (e.g.,  "what  was  spent"  is  a  proxy  for  "what 
should  have  been  spent,"  some  initial  spares  requirements 
were  purchased  with  replenishment  spares  budget  authority, 
etc . ) . 

The  authors  provided  no  discussion  of  the  logic  behind 
the  cost  drivers  selected  other  than  to  say  they  anticipated 
a  positive  relationship  with  the  dependent  variable.  The 
two  CERs  with  the  highest  R^  values  both  involved  the  single 
independent  variable,  "weapon  system  cost."  One  CER  was 
linear  and  the  other  involved  a  logarithmic  transformation 
of  the  dependent  and  independent  variable.  The  authors  made 
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no  conunent  as  to  which  CER  they  felt  most  accurately 
described  the  true  relationship.  This,  they  said,  depended 
upon  "the  analyst's  best  intuition  about  the  underlying 
relationships"  (8).  The  next  model  included  three  linear 
independent  variables — procurement  quantity,  empty  weight, 
and  maximum  speed.  The  final  model  included  these  same 
variables  plus  two  dummy  variables  to  distinguish  attack, 
fighter,  electronic,  and  bomber  aircraft  from  cargo  and 
tanker  aircraft  (8).  Again,  there  is  no  discussion 
concerning  which  relationship  the  authors  preferred,  and  no 
discussion  of  any  model  diagnostics  performed  at  any  time 
during  the  study. 

Literature  Review  Summary 

Throughout  the  course  of  the  literature  review,  one 
factor  stood  out--and  that  was  the  great  number  of 
difficulties  associated  with  developing  spares  CERs.  Upon 
concluding  their  attempts  to  develop  a  better  spares  CER, 
RAND  modelers  summarized  their  feelings:  "In  short,  the 
problem  is  extremely  difficult.  That  was  clear  before,  but 
it  is  even  clearer  now"  (Crawford,  Lansdowne,  and  Finnegan, 
1988:vi).  Data  selection  poses  numerous  problems  for  model 
developers.  Assuming  one  can  even  locate  a  data  source,  one 
is  often  disappointed  to  find  that:  1)  the  data  is  at  the 
wrong  level  of  aggregation,  2)  the  data  covers  a  different 
time  span  than  required  (e.g.,  calendar  year  instead  of 
fiscal  year),  3)  there  are  strategic  holes  in  the  data,  and 
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4)  the  data  is  discredited  by  numerous  other  factors  (e.g., 
the  obligation  data  is  influenced  by  political 
considerations,  is  hard  to  track  to  spares  usage  due  to  its 
multi-year  availability,  etc.). 

An  examination  of  existing  spares  CER  based  models 
provided  ideas  for  cost  driver  candidates  but  little  comfort 
that  a  statistically  significant  model  would  be  found.  The 
models  were  weak  in  this  area  (i.e.,  poor  R^'s,  poor  mean 
standard  errors,  etc.).  The  model  source  documents  were 
also  weak  in  explaining  the  underlying  logic  behind  their 
models.  One  had  no  idea,  for  the  most  part,  if  the 
relationships  found  were  correlational  or  causal  in  nature. 
Additionally,  the  model  source  documents  provided  little,  if 
anything,  in  the  way  of  model  diagnostics  descriptions. 

Although  the  author  cannot  guarantee  that  he  will 
develop  a  statistically  significant  model,  an  attempt  will 
be  made  to  improve  upon  (relative  to  the  models  evaluated  in 
the  literature  review)  the  documentation  of  the  model. 
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1 1 1_j_  Methodol  oav 


Introduction 

The  general  overview  of  the  entire  cost  estimating 
process  (research  objective  number  one)  and  summary-level 
description  of  current  OSS  cost  models  applicable  to  spares 
(research  objective  number  two)  are  purely  descriptive  in 
nature.  This  chapter  will  explain  the  methodology  used  in 
support  of  research  objective  number  three. 

This  third  objective  involved  developing  a  parametric 
model  for  predicting  recoverable  spares  costs.  No  initial 
spares  CER  was  attempted  due  to  a  lack  of  practically 
accessible  data,  as  identified  in  Chapter  II.  The 
replenishment  spares  model  was  accomplished  in  two  parts:  1) 
a' multiple  independent  variable  (MIV)  CER  relating 
condemnation  costs  to  aircraft  physical  and  performance 
characteristics,  and  2)  a  single  independent  variable  (SIV) 
CER  comparing  replenishment  spares  requirements  to 
condemnation  costs.  This  second  part  is,  in  essence,  part 
of  an  attempt  to  develop  a  demand  volatility  factor.  The 
predicted  condemnation  spares  cost  must  be  multiplied  by  its 
associated  demand  volatility  factor  to  arrive  at  the  total 
replenishment  spares  estimate.  Both  MIV  and  SIV  models  were 
developed  with  linear  regression  using  SAS  statistical 
analysis  software  (maintained  on  the  Air  Force  Institute  of 
Technology  (AFIT)  VAX  computer  system).  The  data  used  to 
develop  the  condemnations  CER  is  at  the  "MDS"  level  of 
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aggregation;  while  the  demand  volatility  factor  data  is  at 
the  "MD"  level.  This  chapter  provides  a  detailed 
explanation  of  the  methodology  used  to  create  both  models, 
beginning  first  with  the  condemnations  CER  and  ending  with 
the  demand  volatility  CER. 

Condemnations  CER  Development 

The  following  subsections  discuss  the  specific 
procedures  which  were  used  to  develop  and  validate  the 
condemnations  spares  model.  General  explanations  are  given 
for  some  terms  and  the  reason  for  using  techniques  are 
explained,  but  detailed  explanations  of  the  actual 
regression  techniques  are  not,  Readers  who  are  unfamiliar 
with  linear  regression  will  need  to  refer  to  a  regression 
text  for  further  details.  The  techniques  mentioned  below 
are  from  the  COST  671  (Defense  Cost  Modeling.)  and  COST  672 
(Model  Diagnostics)  coui.'ses  taught  at  AFIT  (Murphy,  1990- 
1991). 

Model  Identification .  The  first  step  in  developing  the 
condemnations  CER  involved  identifying  logical  cost  drivers. 
This  process  is  known  as  model  identification.  In  order  to 
identify  the  underlying  causal  relationships  which  drive  a 
cost  estimate,  one  should  become  familiar  with  the  system(s) 
for  which  the  cost  estimate  is  being  developed.  Given  the 
time  constraints  of  thi ?  thesis,  this  task  was  accomplished 
through  a  review  of  previous  spares  CER  work  and 
consultation  with  spares  experts.  While  one  can  never  be 
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sure  of  capturing  all  the  cost  drivers,  statistical  measures 
(which  will  be  discussed  shortly)  can  be  used  to  determine 
what  portion  of  the  total  error  is  captured  by  the  model. 

A  specific  consideration  under  the  general  "model 
identification"  heading  is  testing  for  interaction  effects 
and  indicator  variables.  Both  were  examined  in  the  model 
development.  If  one  changes  the  value  of  an  independent 
variable  and  the  resulting  change  in  cost  is  dependent  upon 
the  value  of  another  independent  variable,  there  is  an 
"interaction  effect"  between  the  independent  variables.  For 
example,  if  the  change  in  spares  cost  related  to  a  change  in 
an  aircraft's  "maximum  speed"  also  depends  upon  the  "empty 
weight"  of  the  aircraft,  there  is  interaction  between  these 
two  variables.  "Maximum  speed"  and  "empty  weight"  were  in 
fact  tested  for  an  interaction  effect,  along  with  "maximum 
load  factor"  and  "aircraft  length  plus  span."  This  was  done 
by  multiplying  the  variables  against  one  another  in  each  of 
the  above  pairs.  The  resultant  products  became  new 
candidate  independent  variables. 

Indicator  variables  are  used  to  determine  if  the  sample 
population  can  be  divided  into  separate  classes  based  upon 
qualitative  differences.  Indicator  variables  were  included 
to  determine  if  spares  costs  are  related  to  the  following 
categories  of  aircraft:  1)  fighter,  attack,  fighter-bomber, 
or  trainer  aircraft;  or  2)  bomber,  tanker  and  military 
transport . 
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Model  Specification .  Closely  related  to  identifying 


the  cost  drivers  is  the  next  step--hypothesizing  logical 
relationships  between  the  dependent  variable  (cost)  and  the 
independent  variables  (cost  drivers).  This  process  is  known 
as  model  specification.  For  example,  "is  the  relationship 
linear  or  non-linear?" 

There  is  no  simple  way  to  quickly  develop  a  CER  from  a 
list  of  cost  driver  candidates.  Knowing  which  variables  to 
include  in  the  model  and  what  relationships  to  specify 
requires  a  great  deal  of  professional  judgement.  One  should 
ensure  that  a  model  makes  logical  sense.  For  example,  a 
model  may  use  aircraft  weight  as  a  cost  driver  for 
recoverable  spares.  The  logic  behind  this  driver  could  be 
that  increased  weight  implies  more  aircraft  parts  and 
therefore  more  spares  are  required.  However,  the  aircraft 
for  which  spares  costs  are  being  predicted  may  be  heavy 
because  it  is  built  with  heavier,  sturdier  materials.  In 
this  case,  the  model  logic  fails  because  the  heavy  aircraft 
weight  is  not  due  to  more  parts.  Rote  application  of  a  CER, 
without  professional  judgement,  can  lead  to  erroneous 
estimates . 

When  numerous  variables  are  identified  as  potential 
cost  drivers,  one  must  do  one's  best  to  determine  which 
variable  or  set  of  variables  constitute  the  most  logical 
CER.  The  more  candidates  for  inclusion  there  are,  the 
greater  the  number  of  possible  combinations  that  one  is 
tempted  to  evaluate.  The  danger  here  lies  in  the  fact  that 
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simply  by  testing  a  large  niimber  of  liifferent  candidate 
variables  using  different  assorted  transformations,  it  is 
likely  that  one  can  develop  a  statistically  significant 
model  merely  by  chance.  Additionally,  with  no  logical 
expectations  concerning  the  underlyiag  causalities  of  the 
variables,  one  has  no  basis  for  determining  if  a 
statistically  significant  model  is  based  upon  correlation 
between  the  dependent  and  independent  variables,  as  opposed 
to  causality. 

Given  the  pitfalls  associated  with  arbitrarily  testing 
different  combinations  of  variables,  the  author  attempted  to 
identify  expected  relationships  prior  to  running  the  models 
themselves.  It  seemed  logical  that  sizing  variables  such  as 
aircraft  "empty  weight"  and  "length  plus  span"  and  the 
annual  utilization  factors,  "flying  hours,"  "sorties," 
"landings,"  and  "number  of  active  aircraft  in  MDS 
inventory,"  would  have  direct  linee^r  relationships  to 
condemnation  costs.  No  plausible  rationale  could  be  thought 
of  for  either  an  inverse  relationship  or  even  a  direct 
relationship  with  a  changing  slope. 

For  "intensity"  of  utilization  factors  such  as  "flying 
hours  per  aircraft,"  "landings  per  aircraft,"  and  "sorties 
per  aircraft,"  and  perf ormance/techr.ical  complexity  factors 
such  as  "thrust  per  engine,"  "maximum  speed,"  "maximum  climb 
rate  at  sea  level,"  "maximum  load  factor,"  etc.,  it  was  felt 
that  condemnations  costs  would  incre.ase  at  an  increasing 
rate  as  these  factors  increased.  This  type  of  relationship 
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is  typically  tested  with  quadratic  transformations  and/or 
logarithmic  transformations  of  both  the  dependent  and 
independent  variables  ("log-log”  transformations). 

The  "flying  hours  per  sortie"  factor  (mission  profile 
factor)  was  thought  to  cause  condemnation  costs  to  increase 
but  at  a  decreasing  rate.  Given  that  an  aircraft  is 
performing  the  same  type  of  mission  continuously,  performing 
it  for  longer  periods  of  time  will  result  in  increased  wear 
and  tear  on  the  aircraft.  However,  after  a  point,  longer 
m:.ssions  begin  to  reflect  different  types  of  missions.  A 
long  ferry  mission  may  not  cause  as  much  wear  and  tear  as  a 
shorter  mission  practicing  "touch  and  go's."  This  type  of 
relationship  is  typically  modeled  by  raising  the  independent 
variable  to  the  ".5"  or  "-1"  power,  a  log-log 
transformation,  or  a  logarithmic  transformation  of  the  cost 
driver  alone. 

The  aircraft  "flyaway  costs"  is  expected  to  have  a 
direct  relationship  to  condemnations  costs.  More  expensive 
aircraft  means  more  expensive  parts  and  it  is  felt  that  this 
will  outweigh  any  reduction  in  condemnations  due  to  improved 
reliability  (which  may  drive  the  higher  acquisition  cost). 
There  was  insufficient  evidence  to  judge  whether  this  should 
be  a  linear  relationship  or  not. 

Data  Normalization.  Once  the  hypotheses  to  be  tested 
were  developed,  the  raw  data  was  collected  and  then 
normalized.  This  was  required  to  ensure  that  differences  in 
costs  between  different  years  was  not  merely  related  to 
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inflation  effects.  The  dependent  variable  data  obtained 
from  the  Weapon  System  Cost  Retrieval  System  (WSCRS)  was 
already  inflated  to  a  common  base  yesr — FY91.  The 
replenishment  spares  annual  requirements  used  in  the  demand 
volatility  analysis  had  to  be  inflated  to  FY91  dollars  also. 

Linear  Regression.  The  next  step  involved  actually 
using  SAS  to  specify  the  relationship  between  the  dependent 
and  independent  variables  in  mathematical  terms.  SAS  fits 
data  to  a  regression  line  using  the  method  of  least  squares 
best  fit. 

Each  regression  line  is  expressed  in  the  following 
equation  form: 

Yj  =  Bg  +  B|X|j  +  +  •••  +  +  «1  (2) 

(Neter,  Wasserman,  and  Kutner,  1989:229) 

where 

B(j,  B2,...,Bp.|  are  parameters  X^r  X^^, . .  .  ,X^  p.l  are 

known  constants 

are  independent  N(0,  )  i  =  l,...,n 

SAS  can  only  work  with  linear  relationships  and  so  cost 
drivers  with  anticipated  non-linear  relationships  were 
transformed  (e.g.,  setting  the  independent  variable  equal  to 
itself  raised  to  a  power)  so  that  the  relationship  would  be 
linear  as  transformed. 

The  initial  SAS  runs  were  made  using  selected  logical 
linear  independent  variables.  Additional  runs  were  then 
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performed  based  upon  what  logically  made  the  most  sense  and 
the  results  of  this  initial  analysis. 

The  SAS  forward  stepwise  regression  function  evaluates 
how  the  model  statistics  change  as  SAS  adds  each  possible 
independent  variable  one  at  a  time  to  the  model ,  starting 
with  the  most  significant  variable.  This  function  does  not 
guarantee  the  best  model/  however,  and  it  was  used  only  as  a 
check  against  the  final  model  developed. 

Model  Validation.  Having  stated  that  developing  a  CER 
is  not  an  easy  task,  it  is  appropriate  to  discuss  the 
diagnostics  which  can  be  performed  to  check  a  model's 
internal  validity.  Each  SAS  regression  run  provides  an 
analysis  of  variance  (ANOVA)  table  with  many  of  the 
statistics  necessary  to  evaluate  the  model  (e.g.,  R^, 
Adjusted  R^,  F-Value,  etc.).  The  format  of  this  table  is 
provided  in  Table  2. 

Numerous  factors  must  be  evaluated  before  a  final  CER 
is  selected.  One  simple  check  is  to  examine  the  signs  of 
the  parameter  estimates  from  each  ANOVA  table  to  see  if  they 
are  logical.  For  example,  a  negative  sign  would  be  logical 
if  one  had  reason  to  believe  that  the  relationship  between 
the  cost  driver  and  cost  was  inverse. 

Given  a  logical  sign  for  the  parameter  estimate,  a 
number  of  statistics  from  the  ANOVA  table  can  then  be 
examined  together  to  determine  the  overall  predictive 
strength  of  the  model .  The  coefficient  of  determination 
(R^)  tells  one  the  percentage  of  total  squared  error 
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TABLE  2 


AWOVA  Table  Format  (SAS) 


Source 

Degrees 

Sum  of 

Mean 

of 

of 

Squared 

Squared 

Error 

Freedom 

1  Error 

Error  F  Value  P 

Value 

Model 

P-1 

SSR=  2 

MSR=SSR/df  MSR/MSE 

* 

Error 

n-P 

SSE=  Yi 

MSE=SSE/df 

Total 

n-1 

SST=  ]£) 

MST=SST/df 

Root  MSE 

* 

R-squared 

* 

Dep  Mean 

* 

Adj  R-sq 

■it 

C.V. 

* 

Parameter  Estimates 

Parameter 

Standard  T  for  HO: 

Variable 

DF 

Estimate 

Error 

Parameter=0 

Prob>!T! 

Intercept 

* 

* 

* 

* 

Driver#! 

* 

* 

it 

* 

* 

Driver#2 

* 

* 

* 

it 

* 

Driver#P 

* 

* 

* 

* 

Where 

=  the  ith  fitted  value  on  the  regression  line 


y  =  the  mean  of  the  observed  values  in  sample  set 

Y^  =  the  ith  observation  from  the  sample  set 

P  =  the  number  of  parameters  in  the  model 
n  =  the  number  of  observations  in  the  sample  set 


accounted  for  by  the  regression  line.  For  this  thesis,  the 
general  rule  of  thumb  is  that  one  doesn't  v;ant  to  accept  an 
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valu€j  less  than  70%.  A  value  of  80%  or  greater  was 
looked  for  in  the  final  model.  This  statistic  can  be 
misleading,  however,  in  that  it  can  be  artificially  driven 
up  by  increasing  the  number  of  independent  variables  whether 
they  are  true  cost  drivers  or  not.  For  this  reason  the 
adjusted  was  compared  to  the  R'  value.  If  the 
two  values  were  not  within  20%  of  one  another,  it  was 
assumed  that  insignificant  variables  were  impacting  the  R^. 

The  F-test  was  used  to  check  the  statistical 
significance  of  each  overall  model  evaluated  and  the  t-test 
was  used  to  check  on  each  individual  independent  variable's 
significance  given  that  the  other  independent  variables  were 
included  in  the  model  being  tested.  The  probability  values 
associated  with  the  F  and  t-values  were  used  to  determine  at 
what  level  of  confidence  the  variables  could  be  considered 
significant.  Again,  70%  is  a  typical  rule-of-thumb  minimum. 
An  80%  or  better  value  was  sought  in  the  final  model. 

The  P-Value  was  looked  at  to  determine  "the  smallest 
significance  level  at  which  the  null  hypothesis  can  be 
rejected"  (Newbold,  1988:339).  Since  one  wants  to  reject 
the  null  hypothesis  (because  one  believes  a  relationship 
exists  to  begin  with),  obviously  the  smaller  the  P  Value  the 
better. 

The  coefficient  of  variation  (CV)  and  prediction 
intervals  (SAS  has  functions  that  can  print  out  the  95% 
prediction  limit  bounds  and,  for  SIV  models,  plot  these 
bounds  around  the  regression  line)  were  used  to  examine  the 
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model's  external  validity.  The  CV  provides  approximately  a 
70%  prediction  interval  (estimate  plus  or  minus  CV)  at  the 
center  of  the  data.  If  a  natural  logarithmic  transf ormatiiDn 
of  the  dependent  variable  was  used,  the  CV  was  replaced  by 
the  root  mean  squared  error  (MSE)  (which  equates  to  the  CV 

/ 

1  in  a  natural  logarithmic  transformation) .  Since  the  CV  (o : 

root  MSE)  gives  one  a  rough  idea  of  how  well  one  is 
predicting  at  the  center  of  the  data,  where  the  bounds  are 
the  smallest,  a  large  CV  can  end  the  analysis  because  one 
knows  that  it's  not  going  to  get  any  better.  If  the  CV  is 
small,  however,  the  prediction  interval  analysis  can  be  used 
to  examine  how  well  the  model  predicts  over  the  entire 
sample  data  set.  The  larger  the  prediction  intervals,  the 
more  uncomfortable  one  becomes  over  the  model's  predictive 
capability.  If  the  prediction  intervals  seem  to  widen 
dramatically  at  the  outer  edges  of  the  sample  data,  this 
will  be  taken  as  an  indication  that  the  model  may  only  apply 
to  the  range  of  the  current  data  set. 

In  order  to  identify  which  data  points  are  in  the 
center  of  the  data  and  which  are  at  the  outer  edges,  one  can 
examine  the  width  of  the  prediction  limit  bounds  for  each 
data  point.  Those  data  points  with  the  smallest  prediction 
intervals  should  be  relatively  close  to  the  center  of  the 
data,  while  those  with  the  largest  prediction  intervals 
should  be  the  farthest  from  the  center  of  the  data.  When 
conducting  this  analysis  on  a  model  with  a  log-log 
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transformation,  one  should  examine  the  prediction  intervals 
in  logarithmic  space  as  opposed  to  arithmetic  space. 

In  addition  to  the  95%  prediction  intervals  provided  by 
SAS,  70%,  80%  and  90%  prediction  intervals  will  also  be 
derived  (for  the  data  point  closest  to  the  center  of  the 
data)  to  determine  the  impact  of  changing  confidence  levels 
upon  the  model  prediction  intervals.  These  bounds  will  be 
plotted  around  the  predicted  value  to  graphically  depict  how 
the  bounds  increase  as  the  alpha  level  decreases. 

None  of  the  statistics  described  above,  taken  alone, 
can  tell  you  how  "good"  the  model  is.  They  all  must  be 
considered  together  to  evaluate  the  various  models  because, 
for  example,  can  be  artificially  inflated  at  the  expense 
of  the  model's  significance.  Additionally,  these  statistics 
are  only  meaningful  if  one  can  assume  that  the  model  is 
properly  identified  and  specified.  The  diagnostics, 
therefore,  do  not  stop  with  these  statistics. 

Residual  Plots .  Another  check  is  to  see  if  the 
models  are  properly  specified.  SAS  allows  one  to  plot  the 
residuals  (deltas  between  the  observed  and  predicted 
condemnation  costs)  against  the  independent  variable  data. 
Theoretically,  one  can  examine  these  plots  for  patterns  in 
the  data  and  if  all  the  residual  plots  appear  to  be 
completely  random,  then  one  may  assume  that  the  model  is 
properly  specified  and  no  further  action  is  required.  If 
even  one  plot  shows  a  pattern,  this  indicates  that  the  data 
is  not  linear  as  transformed  and  thus  a  (different) 
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transformation  is  required.  If  and  when  this  is  the  case, 
two  other  SAS  functions  can  be  used  in  combination  with 
these  basic  residual  plots  to  determine  which 
transformations  are  required.  Partial  regression  plots  and 
residuals  plotted  against  important  independent  variables 
omitted  from  the  model  can  also  be  used  to  identify 
necessary  transformations.  This  "theoretical"  procedure  was 
not  followed  in  this  thesis,  however,  because  patterns  in 
the  data  were  expected  but  not  found.  Rather  than  stopping 
at  this  point,  the  author  vrent  with  his  professional 
judgement  and  continued  with  the  partial  regression  plots 
and  omitted  variable  plots,  hoping  to  identify  necessary 
transformations.  Each  time  a  variable  was  transformed,  the 
process  was  started  all  over  with  plotting  the  residuals  for 
every  independent  variable  and  checking  for  patterns  in  the 
data.  Additionally,  transformations  which  made  logical 
sense  were  evaluated,  even  in  the  residual  plots  did  not 
indicate  that  the  transformations  were  necessary. 

Outliers.  Still  another  check  involves 
identifying  outliers  in  the  sample  data.  SAS  has  a  function 
which  prints  out  leverage  values  (measurements  of  an 
observation's  distance  from  the  center  of  the  data)  which 
can  be  used  to  identify  outliers  with  respect  to  X.  If  a 
leverage  value  (h^j)  is  greater  than  twice  the  number  of 
parameters  (P)  divided  by  the  number  of  data  points  (n), 
than  the  data  point  is  an  outlier.  If  P  is  large  relative 
to  n,  then  this  formula's  value  may  exceed  one.  If  this  is 
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the  case,  the  rule  of  thumb  value  is  switched  to  .5  for  the 
maximum  acceptable  h^. 

"Internally  studentized  residuals"  were  used  to 
determine  outliers  with  respect  to  Y.  The  following 
equation  can  be  used  to  produce  internally  studentized 
residual  values: 

e  /  [MSE(1  -  (3) 

If  this  value  exceeds  the  t-value  associated  with  10%  alpha 
and  n  -  P  degrees  of  freedom,  the  data  point  is  an  outlier. 
SAS  has  a  function  which  prints  out  the  internally 
studentized  residuals 

There  are  numerous  possible  causes  for  these  extreme 
values  in  the  data: 

1)  The  outlier  date,  point  belongs  to  a  different 
population. 

2)  The  independent  variable  data  includes  measurement 
error. 

3)  There  was  some  unique  major  event  with  significant 
impact  on  the  data. 

4)  Significant  independent  variables  are  excluded  from 
the  model . 

5)  The  data  point  just  so  happens  to  fall  in  a  tail  of 
the  data  distri))ution. 

6)  The  model  is  imi-roperly  specified. 

7)  The  error  terms  come  from  non-normal  distributions. 
What  one  decides  t('  do  about  the  outliers  depends  upon 

their  cause(s)  and  thei)  influence.  If  one  determines  that 
an  outlier  belongs  to  ar other  population,  it  can  be  thrown 
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out.  If  there  are  an  insignificant  number  of  outliers 
compared  to  the  sample  size,  they  can  be  thrown  out.  If  the 
outliers  are  needed,  due  to  the  sample  size,  and  they  are 
non-inf luential ,  they  can  be  left  alone.  If  they  are  needed 
and  influential,  an  attempt  must  be  made  to  adjust  the 
outliers  to  minimize  their  effect  on  the  model. 

To  determine  if  an  outlier  is  influential,  SAS  produces 
three  measures  of  inf luential lity :  1)  DFFITS,  2)  DFBETAS, 
and  3)  Cook's  Distance. 

DFFITS  values  measure  the  influence  of  "case  i"  on  the 
"fitted  value"  of  condemnation  costs  and  it  can  be 
calculated  with  the  following  equation: 

(DFFITS)i  =  (h^/(l  -  hji))’’  (4) 

(Neter,  Wasserman,  and  Kutner,  1989:401) 


The  rule  of  thumb  for  this  equation  is  that  the  absolute 
value  of  DFFITS  may  not  exceed  one  for  small  to  medium  sized 
data  sets  and  2(P/n)‘^  for  large  data  sets. 

DFBETAS  values  measure  "the  influence  of  the  ith  case 
on  each  regression  coefficient  bj^"  and  can  be  calculated  as 
foil ows : 


( DFBETAS )bj^  =  bj  -  bi.(p  /  (5) 

(Neter,  Wasserman,  and  Kutner,  1989:402) 


where 


Cj^j^  is  the  kth  diagonal  element  of  (X'X)'^ 
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The  rule  of  thumb  for  DFBETAS  is  that  its  absolute  value  may 
not  exceed  2/(n)‘^. 

Cook's  Distance  is  "an  overall  measure  of  the  combined 
impact  of  ith  case  on  all  of  the  estimated  regression 
coefficients"  and  it  can  be  calculated  with  the  following 
equation: 


Di  =  (b  -  b|ij)'X*X(b  -  bfij)  /  pMSE  (6) 

(Neter,  Wasserman,  and  Kutner,  1989:403) 

where 

b  is  the  vector  of  estimated  regression  coefficients 
obtained  when  all  n  data  points  are  used. 

b^j  is  the  vector  obtained  when  the  ith  case  is 
omitted. 

The  rule  of  thumb  for  Cook's  Distance  can  be  found  by 
comparing  it  to  the  P  di:3tribution  value  for  the  fiftieth 
percentile  with  numerator  degrees  of  freedom  equal  to  the 
number  of  regression  coefficients  (p)  (including  the 
intercept  term)  and  denominator  degrees  of  freedom  equal  to 
n  -  p.  If  the  Cook's  Distance  value  exceeds  the  F 
distribution  value  the  point  is  considered  an  influential 
outlier. 

Mul ticol 1 inearitv .  Multicollinearity  is  a 
violation  of  the  assumed  independence  between  the 
independent  variables.  As  multicollinearity  is  introduced 
to  a  model  the  variance  of  the  regression  coefficients 
become  very  large  and  the  coefficient  values  themselves 
become  highly  unstable.  They  may  even  take  on  the  wrong 
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sign.  As  a  result,  the  predicted  cosfficient  values  are 
more  likely  to  be  further  from  the  true  population  parameter 
values  and  the  variables'  t-values  may  get  smaller. 

SAS  provides  several  functions  to  test  for 
mul ticol 1 inearity  in  addition  to  the  symptoms  mentioned 
above.  Pairwise  correlation  matrices,  tolerance  values  and 
the  SAS  COLLINOINT  function  can  be  used.  A  pairwise 
correlation  matrix  shows  the  collinearity  between  two 
variables.  The  tolerance  values  depict  multicollinearity 
between  a  variable  and  the  rest  of  tie  model.  The 
COLLINOINT  function  is  a  good  method  to  use  if  the  number  of 
cost  drivers  exceeds  four.  For  each  variable  a  "condition 
number"  is  given.  If  this  number  exceeds  ten,  then  a 
significant  degree  of  multicollinearity  is  present.  After 
identifying  a  variable  with  multicollinearity,  one  looks  to 
the  "variance  proportions"  listed  fcr  each  remaining  cost 
driver  to  determine  what  variables  are  contributing  to  the 
mul ticol 1 inearity . 

Heteroscedasticitv .  Hetei oscedasticity  is  a 
violation  of  the  assumption  that  the  error  terms  come  rom 
distributions  with  constant  variance.  To  check  for  th-s 
problem,  the  residuals  can  be  plotted  against  the  expected 
value  of  condemnation  costs.  If  het eroscedasticity  is 
present,  the  residual  pattern  will  either  converge  or 
diverge  as  the  expected  condemnatior  cost  values  increase. 

If  the  spread  of  residuals  remains  relatively  constant,  one 
may  assume  that  heteroscedasticity  is  not  present. 
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Model  Predictive  Capability .  After  completing  the 


diagnostic  checks  described  above,  the  best  linear  model, 
the  best  arithmetic  transformation  model  and  the  two  best 
log-log  transformation  models  were  used  to  predict  the 
condemnation  costs  for  an  MDS  database  excluded  (for  this 
purpose)  from  the  original  data  set.  The  predicted  values 
v;ere  compared  to  the  actual  values  to  see  how  well  the 
models  performed. 

Model  Sensitivity .  The  best  linear  model,  best 
arithmetic  transformation  model  and  two  best  log-log 
transformation  models  were  also  tested  to  see  how  sensitive 
they  were  to  changes  in  the  values  of  independent  variables. 
This  was  accomplished  by  first  using  the  models  to  predict 
condemnation  costs  for  several  aircraft  MDS.  Then  new 
estimates  were  created  after  increasing  the  value  of  one 
independent  variable  in  each  MDS  by  twenty  percent  (and 
leaving  the  other  independent  variable  values  alone).  After 
repeating  this  action  for  each  of  the  models'  independent 
variables  and  noting  the  impacts  on  the  condemnation  costs 
estimates,  the  process  was  repeated  except  that  the 
independent  variable  values  were  decreased  by  twenty 
percent . 

Smal 1  Database  Performance.  Only  those 
independent  variables  dealing  with  component  utilization 
changed  over  time  (e.g.,  sorties,  number  of  aircraft  in  MDS 
inventory).  It  was  anticipated  that  reducing  the  database 
by  replacing  these  annual  variable  values  (five  data  points 
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for  most  of  the  variables)  with  their  MDS  averages  might 
provide  a  more  statistically  significant  model  if  the 
individual  values  varied  greatly  within  an  MDS.  However, 
this  smoothing  effect  might  also  lead  to  unrealistic 
confidence  in  the  models'  ability  to  predict  an  annual 
spares  requirement.  This  database  smoothing  technique  was 
tested  as  the  final  analysis  step. 

Demand  Volatility  Factor  CER 

The  SIV  model  for  demand  volatility  was  developed  by 
regressing  annual  replenishment  spares  requirements  data 
against  annual  condemnation  costs  for  the  same  MD  data  set. 
By  restricting  the  Y-intercept  to  equal  zero,  the  demand 
volatility  factor  is  simply  the  value  of  the  regression 
coefficient  from  the  resulting  CER.  In  order  to  look  at  the 
statistical  measures  associated  with  the  CER,  another  model 
K  s  created  with  no  restriction  on  the  Y-intercept.  No 
logical  explanation  was  found  for  any  relationship  between 
replenishment  spares  and  condemnations  other  than  a  direct 
linear  one  and  therefore  no  transformations  of  the 
condemnations  data  were  considered. 

The  demand  volatility  factor  currently  used  in  the  HQ 
AFLC/FMC  Logistic  Support  Cost  model  was  arrived  at  by 
comparing  condemnations  with  replenishment  spares 
obligations  and  simply  averaging  the  various  MDS  ratios  to 
come  up  with  a  standard  factor.  As  mentioned  in  Chapter  II, 
there  are  numerous  factors  which  discredit  the  use  of 
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obligations  data  in  model  development.  Using  scrubbed 
replenishment  spares  requirements  generated  by  the  D041 
system  is  an  improvement  over  the  obligation  data. 
Additionally,  using  SAS  to  derive  the  CER  provided 
information  on  the  statistical  significance  of  the 
relationship--something  not  found  in  a  straight  ratio  of 
replenishment  spares  obligations  to  condemnation  costs. 

In  addition  to  the  SAS  produced  CER,  however,  annual 
demand  volatility  factors  were  calculated  by  dividing  annual 
MDS  replenishment  spares  requirements  oy  annual  MDS 
condemnations  costs.  These  results  were  analyzed  for  any 
year-to-year  trends  in  MDS  specific  demand  volatility 
factors.  Additionally,  demand  volatility  factors  were 
considered  for  different  mission  categories  (e.g.,  fighter, 
bomber,  etc.). 

Before  the  replenishment  spares  and  condemnations  data 
could  be  compared,  however,  there  were  several  steps  which 
had  to  be  taken  to  ensure  (as  much  as  possible)  that  apples 
were  being  compared  to  apples,  and  not  oranges.  For 
example,  it  was  already  mentioned  that  the  Weapon  System 
Cost  Retrieval  System  (WSCRS)  condemnation  data  was  provided 
in  FY  91  dollars  and  that  the  scrubbed  D041  replenishment 
spares  requirements  had  to  be  inflated  from  then-year  to  FY 
91  dollars.  The  WSCRS  system  inflates  prior  year 
condemnations  to  FY  91  dollars  using  separate  escalation 
rates  (obtained  from  APR  173-13)  for  engine  material, 
avionics  material,  and  airframe  material  (AFLCM  173-264, 
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1990:136).  Because  the  replenishment  spares  requirements 
data  was  aggregated  at  the  MD  level,  the  best  that  could  be 
done  was  to  inflate  the  data  using  the  3080  Other 
Procurement  inflation  rates  (also  obtained  from  APR  173-13). 

Two  additional  factors  had  to  be  accounted  for  in  order 
to  compare  the  replenishment  and  condemnation  spares 
requirements  data  sets.  First,  the  scrubbed  D041 
replenishment  spares  requirements  included  separate 
categories  for  "common  spares,"  and  the  FlOO  and  FllO  engine 
spares  (used  in  the  F-15  and  F-16  aircraft);  while  the  WSCRS 
system  had  already  distributed  the  condemnations  costs  for 
these  categories  among  the  relevant  MDS. 

The  WSCRS  system  can  split  out  for  each  MD  what  percent 
of  the  annual  condemnations  costs  are  MD  unique  and  what 
percent  are  common  to  other  MDs.  An  attempt  was  made  to 
distribute  the  scrubbed  D041  replenishment  spares  "common 
spares"  category  using  these  WSCRS  MD  percentages.  For 
example,  if  the  WSCRS  system  showed  that  fifty  percent  of  an 
MD's  condemnation  costs  were  MD  unique,  it  was  assumed  that 
the  replenishment  spares  requirement  for  that  MD  represented 
only  fifty  percent  of  the  total  replenishment  spares 
requirement  also,  and  the  remaining  fifty  percent  must  be 
included  in  the  "common  spares"  requirements  pool.  It 
became  apparent  that  there  was  a  definitional  difference  in 
the  term  "common  spares"  between  the  D041  and  WSCRS  systems. 
Using  the  WSCRS  ratios  suggested  that  annual  common  spares 
allocations  should  be  significantly  greater  than  the  annual 
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conunon  spares  pools  identified.  Given  no  clear 
understanding  of  the  exact  problem,  this  methodology  was 
abandoned . 

The  HQ  APLC  Recoverable  Spares  Stock  Fund  Manager 
indicated  that  his  office  used  factors  developed  by  the 
Logistics  Management  Ins“itute  (LMI)  to  spread  the  common 
spares  among  the  various  aircraft  MD  when  necessary.  The 
LMI  factors  are  used,  not  because  they  are  believed  to  be 
perfect  by  any  means,  but  because  they  represent  the  best 
methodology  available  (Rosenthal,  1991).  The  LMI  factors 
provide  the  percentage  of  the  common  spares  requirements 
attributable  to  each  MD  based  upon  quarterly  MD  buy 
requirements  for  the  common  components.  The  factors  change 
from  year  to  year  and  even  from  quarter  to  quarter  depending 
upon  the  current  status  and  projected  buy  requirements. 

These  factors  were  used  to  allocate  the  common  spares 
to  the  individual  MDs.  It  should  be  noted  that  of  the  seven 
sets  of  factors  used  (one  for  each  data  year),  four  sets 
were  based  upon  end-of-March  databases  which  estimated  MD 
percentages  for  the  same  fiscal  year  (e.g.,  FY88  percentages 
were  based  upon  end-of-March  FY88  status);  while  another  two 
sets  used  end- of -Sept ember  status  to  project  percentages  for 
the  following  fiscal  year  (e.g.,  the  FY82  percentages  were 
developed  based  upon  end-of-September  FY81  status).  The 
final  factor  set  used  end-of-March  status  in  FY84  to  predict 
the  FY85  percentages.  The  fact  that  different  quarters  were 
used  and  that,  in  three  cases,  the  percentages  were 
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projected  a  year  into  the  future,  introduced  additional 
uncertainty  to  the  identification  of  trends  between  the 
fiscal  years. 

In  order  to  spread  the  annual  FlOO  and  FllO 
replenishment  spares  requirements  between  the  F-15  and  F-16, 
annual  engine  flying  hour  data  from  the  WSCRS  system  was 
used.  This  data  set  divided  the  total  annual  engine  flying 
hours  for  each  engine  type  between  tie  two  MDs .  It  was 
assumed  that  the  number  of  engine  f 1  /ing  hours  for  a 
particular  engine  drove  the  same  rat;.o  of  replenishment 
spares  in  the  F-15  as  it  did  in  the  ^-16.  For  example,  if 
the  F-15  was  accountable  for  seventy  five  percent  of  the 
total  engine  flying  hours,  then  it  also  received  seventy 
five  percent  of  the  engine's  replenishment  spares 
requirements  pool. 

The  second  factor  which  had  to  5e  accounted  for  was  the 
fact  that  replenishment  spares  are  p  irchased  lead  time  away 
from  the  year  in  which  they  will  be  ictually  used  (according 
to  predicted  usage).  Therefore  the  'eplenishment  spares 
requirements  should  have  been  compared  to  the  associated  MD 
condemnation  costs  lead  time  away.  For  this  analysis,  an 
average  lead  time  of  two  years  was  used  to  offset  the 
replenishment  spares  requirements  and  condemnation  costs 
data  sets.  For  example,  FY  1982  replenishment  spares 
requirements  were  compared  to  FY  1984  condemnation  costs. 

In  order  to  smooth  the  error  introduced  by  this  approach,  an 
additional  comparison  was  made  between  the  total  (all  FYs) 
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condemnations  costs  for  each  MD  and  the  associated  total 
(all  FYs)  replenishment  spares  requirements. 

Meeting  the  Research  Objectives 

In  designing  the  condemnations  cost  model , 
relationships  between  cost  and  potential  cost  drivers  taken 
from  aircraft  performance  and  physical  characteristics  were 
evaluated.  It  was  thought  that  these  relationships  could 
offer  a  great  improvement  over  historical  factor-based 
models  using  aircraft  flyaway  costs.  They  offer  insight 
into  those  factors  which  actually  "drive"  the  cost  of 
recoverable  spares. 

The  condemnations  cost  model  had  to  pass  the 
diagnostics  tests  described  earlier  if  it  was  to  be  accepted 
as  a  valid  cost  prediction  model.  In  reality,  one  should 
never  expect  to  obtain  a  "perfect  model"  that  passes  all 
tests  with  flying  colors.  For  example,  multicollinearity  is 
a  very  common  issue  because  the  cost  drivers  being  evaluated 
are  all  common  to  the  weapon  system  for  which  the  model  is 
being  designed.  This  commonality  breeds  multicollinearity! 
In  the  end,  the  analyst  must  use  his  or  her  professional 
judgement  in  compromising  between  the  various  diagnostics 
results  to  come  up  with  a  reasonable  model.  The  general 
criteria  which  were  established  as  goals  for  the  research 
are  summarized  below: 

1)  The  models*  should  be  greater  than  or  equal  to 
8p%.  Their  adjusted  R^  is  should  be  within  20%  of 
H.  • 
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2)  Confidence  levels  for  the  models'  significance 
should  be  greater  than  or  equal  to  80%. 

3)  P-values  should  be  relatively  small  (no  rule  of 
thumb) . 

4)  Prediction  intervals  should  not  widen  dramatically 
at  outer  bounds  of  the  sample  distribution. 

5)  Residual  plots  for  the  cost  drivers  should  be 
random. 

6)  Outliers  should  be  minimal  and  their  influence 
reduced  through  proper  data  adjustment. 

7)  The  impact  of  multicollinearity  should  be  minimal. 

The  demand  volatility  factor  CER  was  developed  to  see 
if  a  statistically  significant  SIV  model  could  be  developed 
using  condemnation  costs  as  the  independent  variable.  The 
same  seven  criteria  listed  above  were  applicable  to  this 
model . 
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IV .  Analysis  and  Findings 


Introduction 

This  chapter  presents  the  analysis  and  findings 
generated  by  the  procedures  described  in  Chaptc^r  III, 
"Methodology."  The  discussion  is  divided  into  two  main 
sections  beginning  with  the  condemnation  CERs  analysis  and 
ending  with  the  demand  volatility  factor  analysis.  Within 
the  condemnation  CERs  section,  the  results  from  four  models 
are  presented:  the  best  lineai  model,  the  best  arithmetic 
transformation  model,  the  second  best  log-log  transformation 
model,  and  the  best  log-log  transformation  model.  The 
demand  volatility  factor  section  contains  both  the  SAS 
generated  SIV  model  results  and  analysis  of  spreadsheet 
generated  demand  volatility  factors. 

Condemnation  CERs 

The  four  models  in  this  section  were  developed  using 
the  procedures  described  in  Chapter  III,  "Methodology." 
Instead  of  providing  a  single  "best"  model,  the  best  linear, 
arithmetic  transformation  and  log-log  transformation  models 
are  presented.  Although  evaluations  are  not  presented  for 
all  the  models  tested  in  reaching  the  best  model  (the  second 
log-log  transformation  model  in  this  case),  these  four 
models  show  how  the  model  performance  changed  as  different 
types  of  transformations  were  attempted. 
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Tables  3  and  4  provide  a  condensed  look  at  the  data 
used  in  the  development  of  these  models.  While  the  data  in 

Table  3 

MDS  Averages  for  Annual  Data 


AVG 

AVG 

AVG 

AVG 

ANNUAL 

ANNUAL 

AVG 

ANNUAL 

ANNUAL 

CONDEMN 

#  OF 

ANNUAL 

FLY  HRS 

#  OF 

MDS 

COSTS 

MDS 

FLY  HR^ 

PER  MDS 

SORTIES 

F15A 

47,134,882 

317 

72,583 

229.25 

F15B 

8,797,006 

54 

13,6C2 

253.82 

F16A 

61,525,807 

606 

169,8:.5 

280.70 

122,246 

F4D 

17,702,001 

468 

98,6:  8 

210.75 

71,363 

F4E 

18,968,917 

608 

145,1  1 

238.19 

110,123 

A7D 

12,667,978 

389 

95,8(-4 

246.68 

59,341 

A7K 

1,110,592 

30 

7, 5:. 3 

249.34 

2,826 

AlOA 

21,580,794 

654 

220,953 

338.07 

105,629 

FlllD 

14,226,418 

85 

17,0:.5 

201.17 

7,476 

T37B 

5,830,023 

689 

272,1  11 

397.02 

196,644 

T38A 

14,560,060 

897 

334,2  J8 

373.80 

248,339 

B52G 

49,673,391 

173 

65,5  iO 

378.83 

8,980 

B52H 

20,689,628 

96 

36,523 

378.88 

4,842 

C5A 

41,076,959 

77 

46,674 

606.59 

8,537 

C130B 

3,393,740 

94 

37,355 

397.51 

17,466 

C130E 

14,515,170 

296 

163,935 

556.65 

85,617 

C141B 

48,950,346 

268 

284,354 

1,062.72 

68,047 

C135A 

62,641,457 

599 

200,259 

333.99 

45,669 

FBlllA 

13,878,724 

69 

17,335 

252.43 

4,832 

Table  3  provide  MDS  averages  for  variables  which  change  over 
time,  annual  (fiscal  year)  data  was  used  in  the  actual  model 
development.  Appendix  B  provides  the  data  as  it  was  used  in 
the  model  development. 

Best  Linear  Model .  None  of  the  linear  models  tested 
provided  very  good  statistical  results.  Among  the  many 
problems  identified  in  the  models,  a  universal  issue  was 
that  the  criterion  for  (80%+)  was  not  achieved. 
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MDS  Data  Used  in  Model  Development 
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Additionally,  C.V.s  were  universally  high  (40%+).  Despite 
these  less  than  awe  inspiring  results,  the  "best"  linear  CER 
is  provided  for  comparison  with  the  transformed  models.  This 
CER  is  expressed  in  the  following  equation: 

CONDEMN  =  -103,454,577  +  149 . 20001( SORT)  +  1,047.701346 

(THRUST)  +  45,649,536(TOEMPWT)  (7) 

where 

CONDEMN  =  annual  condemnations  requirement  ($) 

SORT  =  MDS  annual  sorties 

THRUST  =  maximum  thrust  per  engine  (lb) 

TOEMPWT  =  maximum  takeoff  weight  (lb) /maximum 
empty  weight  (lb) 

An  immediate  flaw  in  this  CER  is  fairly  obvious.  The 
large,  negative  Y-intercept  term  allows  this  model  to 
predict  negative  annual  condemnation  requirements  for  those 
MDS  with  small  cost  driver  inputs  (particularly  those  with 
small  TOEMPWT).  Additional  statistical  information  can  be 
found  in  Table  5,  which  provides  the  SAS  analysis  of 
variance  (ANOVA)  results  for  this  CER. 

Looking  at  the  positive  attributes  of  this  model,  each 
of  the  cost  drivers  are  significant  to  the  99.9%  level  and 
possess  the  correct  sign.  The  adjusted  R  is  close  to  the 
value,  indicating,  once  again,  that  all  the  variables  are 
making  significant  contributions  to  the  model  and  aren't 
included  simply  to  drive  up  the  R^.  Additionally,  the 
overall  model  has  a  very  low  P-Value  (.0001).  Finally,  the 
model  is  parsimonious  in  its  selection  of  cost  drivers. 
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Table  5 


Linear  Model  Analysis  of  Variance 


Source 

DF 

Sum  of 
Squares 

Mean 

Square 

F  Value  Prob>F 

Model 

3 

2.0833471E16 

6.9444903E15 

57.013  0.0001 

Error 

66 

8.0391021E15 

1.2180458E14 

C  Total 

69 

2.8872573E16 

Root  MSE  11036511.0996  R-square  0.7216 

Dep  Mean  26944355.4429  Adj  R-sq  0.7089 

C.V.  40.96038 

Parameter  Estimates 

Parameter  Standard  T  for  HO: 

Variable  DF  Estimate  Error  Parameter=0  Prob>!Tj 

INTERCEP  1  -103454572  10464398.517  -9.886  0.0001 

SORT  1  149.200010  22.42411015  6.654  0.0001 

THRUST  1  1047.701346  180.16734946  5.815  0.0001 

TOEMPWT  1  45649536  4066074.0692  11.227  0.0001 

Unfortunately,  the  model  has  a  more  extensive  list  of 
negative  attributes.  Although  the  model  appears  to  be 
significant  and  parsimonious,  its  R  value  (.7216)  does  not 
pass  the  model  acceptance  criterion.  This  low  value  implies 
that  there  may  be  other  sources  of  error  not  accounted  for 
by  the  model . 

The  C.V.  value  (40.96038%)  doesn't  instill  confidence 
in  the  model's  predictive  accuracy.  Since  one  knows  that 
the  prediction  interval  will  only  get  wider  as  one  attempts 
to  predict  farther  from  the  center  of  the  data,  one  would  be 
justified  in  stopping  at  this  point.  For  completeness  of 
analysis,  however,  the  prediction  intervals  for  every  data 
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point  were  examined  to  record  how  the  interval  widths  varied 
across  the  data.  The  smallest  bound  (using  the  SAS  provided 
95%  bounds)  for  the  data  set  was  plus  or  minus  $22,249,322. 

The  largest  bound  in  the  data  set  was  plus  or  minus 
$24,161,129.  One  can  infer  from  these  numbers  that, 
percentage  wise,  the  bounds  do  not  increase  dramatically  as 
one  moves  from  the  center  of  the  data  to  the  outer  edges  of 
the  data.  Unfortunately,  the  interval  vras  poor  to  begin 
with,  so  this  information  provides  little  benefit. 

Another  test  was  performed  to  determine  how  the  model's 
prediction  intervals  were  impacted  by  varying  the  level  of 
confidence  used  in  the  model  development.  Figure  1  shows 
the  predicted  condemnations  cost  for  the  datapoint  closest 
to  the  center  of  the  data  (i.e.,  the  datapoint  with  the 
smallest  prediction  interval)  and  it!  associated  upper  and 
lower  bounds  for  four  different  levels  of  confidence  (70%, 

80%,  90%,  and  95%). 

This  figure  allows  one  to  see  tlie  magnitude  of 
prediction  interval  growth  as  one’s  -lonfidence  level  moves 
from  70%  (plus  or  minus  $11,641,289)  to  95%  (plus  or  minus 
$22,249,322).  If  one  is  attempting  -o  predict  costs  for  a 
datapoint  away  from  the  center  of  th -■  data,  and  do  so  with  a 
high  confidence  ^evel,  one  can  expec.  a  very  large 
prediction  interval . 

The  ANOVA  based  analysis  was  only  the  beginning  of  the 
diagnostics  performed  on  each  CER.  The  following 

subsections  address  the  complete  line  of  diagnostic  checks  performed. 
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Predicted  Value  Lower  Bound  Upper  Bound 


Figure  1.  Linear  Model  Confidence  Level  Sensitivity 

Residual  Plots.  For  each  cost  driver,  data  point 
residuals  were  plotted  against  their  corresponding  cost 
driver  values  in  an  attempt  to  identify  necessary  variable 
transformations.  In  every  case  tested,  the  plots  appeared 
to  be  randomly  scattered.  Normally  this  is  exactly  the 
results  one  hopes  to  achieve,  because  it  indicates  that  the 
variables  are  properly  specified.  It  was  anticipated  that 
variables  such  as  "number  of  sorties"  and  "number  of 
aircraft"  would  have  direct  linear  relationships;  but  for 
those  variaoles  dealing  with  the  intensity  of  parts 
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utilization  and  parts  performance,  transformations  were 
logically  expected.  Therefore  the  "random  plots"  were 
attributed  to  noise  in  the  data  and  two  additional  residual 
plot  tests  were  performed:  1)  partial  regression  plots  and 
2)  omitted  variable  residual  plots. 

For  these  plots,  one  hopes  to  see  a  pattern  in  the 
residuals,  because  the  pattern  (be  it  linear,  or  some  sort 
of  non-linear  pattern)  signifies  that  the  independent 
variable  does,  in  fact,  have  a  relationship  with  the 
dependent  variable.  Two  variables  were  removed  from  the 
"best"  linear  model  because  these  residual  plots  show  no 
patterns  in  the  data.  This  action  was  also  confirmed  by 
improvement  in  the  model's  significance.  The  only  non¬ 
linear  pattern  identified  is  that  reproduced  in  Figure  2. 

The  partial  regression  residual  plot  for  tl  i  variable 
"maximum  takeoff  weight /maximum  empty  weight  (TOEMPWT)" 
displays  residuals  increasing  at  an  increasing  rate  as  the 
TOEMPWT  values  increase.  The  KC-135  data  points  at  the 
extreme  right  in  the  graph  appear  to  compose  a  separate 
grouping  from  the  rest  of  the  data  points.  The  nature  of 
the  cargo  and  mission  (aerial  refue'ing)  for  this  aircraft 
may  provide  unique  design  considerations  that  make  this  MBS 
an  outlier  with  respect  to  the  Y  variable.  The  pattern  seen 
in  Figure  2  suggests  that  TOEMPWT  should  either  be 
transformed  using  a  positive  exponent  greater  than  one  (an 
exponent  value  of  two  was  used)  or  a  log-log  transformation 
should  be  used  for  the  entire  model. 
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Outliers .  The  KC-135  was  not  the  only  MDS  with 
datapoints  that  were  outliers.  Seveiiteen  of  the  seventy 
data  points  had  studentized  residual  values  which  classified 
them  as  outliers  with  respect  to  Y  (the  cutoff  value  being 
approximately  1.296).  The  MDS  represented  by  these  outliers 
include  the  F-16A,  the  B-52G  and  H  models,  the  C-5A,  the  C- 
130E,  the  C-141B  and  the  KC-135A.  The  leverage  values  for 
six  datapoints  indicated  that  these  MDS  were  outliers  with 
respect  to  X  (the  cutoff  value  being  .1143).  The  MDS 
represented  by  these  outliers  include  the  T-38A  and  the  C- 
5A. 

Additional  tests  were  run  to  determine  how  influential 
these  outliers  were.  The  results  varied  depending  upon  the 
test  used.  According  to  the  Dffits  test,  five  data  points 
representing  three  MDS  (F-16A,  C-5A,  and  KC-135A)  were 
identified  as  influential  outliers  with  respect  to  Y  (the 
cutoff  value  being  .47804).  Though  this  is  a  significant 
reduction  from  the  earlier  figure  of  seventeen,  the  number 
is  still  too  high.  By  definition,  outliers  should  represent 
an  abnormality,  not  a  common  occurance.  The  Dfbeta  test 
showed  that  four  F-16A  datapoints,  one  T-38A  datapoint,  two 
C-5A  datapoints,  one  C-130B  datapoint  and  three  KC-135A 
datapoints  were  influential  outliers  (the  cutoff  value  being 
.2390).  The  final  inf luential lity  test.  Cook’s  D,  failed  to 
identify  any  influential  outliers. 

Typically,  one  doesn't,  expect  to  see  a  large  proportion 
of  outliers  within  a  data  set.  Outliers,  by  definition. 
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should  not  be  the  norm.  However  in  this  model  a  large 
number  are  identified.  This  phenomenon  is  at  least 
partially  explained  by  the  wide  variety  of  MDS  represented 
in  the  model  database.  In  attempting  to  develop  an  all- 
inclusive  CER  for  aircraft  condemnations  requirements,  noisy 
data  became  a  byproduct  of  the  model's  generalizabllity . 

After  identifying  the  outliers  the  next  issue  was  how 
to  deal  with  them.  According  to  acceptance  criterion  number 
six,  the  number  of  outliers  should  be  "minimal"  and  their 
impact  should  be  minimized  through  proper  data  adjustment 
techniques.  However,  in  this  case  the  broad  population 
definition  (all  Air  Force  MDS  meeting  the  "mature"  weapon 
system  criteria)  made  it  impossible  to  simply  drop 
datapoints  for  being  outside  the  population.  Additionally, 
no  information  was  found  which  supported  any  type  of  data 
adjustment.  Therefore,  no  action  was  taken  to  remove  or 
adjust  the  outliers. 

Multicol linearity.  Both  tolerance  values  and  the 
SAS  COLLINOINT  function  were  evaluated  to  determine  the 
impact  of  multicollinearity  upon  the  model.  Neither  test 
showed  any  sign  of  a  significant  problem.  The  minimum 
tolerance  value  was  .64698110  (with  the  cutoff  being  .10) 
and  the  maximum  condition  number  was  2.00869  (with  the 
cutoff  being  10.0). 

Heteroscedasticitv .  When  datapoint  residuals  were 
plotted  against  predict'd  condemnation  values  (see  Figure 
3),  a  diverging  pattern  in  the  data  indicated  the  presence 
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of  heteroscedasticity .  Given  the  large  number  of  problems 
already  identified  with  this  model,  no  attempt  was  made  to 
correct  for  the  apparent  heteroscedasticity. 


Model  Predictive  Capability.  Table  6  shows  how 
well  the  best  linear  model  performed  in  predicting  costs  for 
datapoints  outside  the  development  database  (see  Appendix  E 
for  validation  data  set). 

TABLE  6 

Linear  Model  Validation  Test  Results 


ACTUAL 

BEST 

Estimate 

CONDEMN 

LINEAR 

Divided  by 

MDS 

COSTS 

ESTIMATE 

ACTUAL 

F16A 

27,537,555 

39,829,790 

1.45 

F4D 

24,151,890 

18,676,955 

0.77 

F4E 

24,254,355 

21,647,671 

0.89 

A7D 

20,075,237 

10,814,153 

0.54 

A7K 

1,560,822 

2,477,468 

1.59 

AlOA 

12,012,017 

30,443,456 

2.53 

FlllD 

14,831,043 

16,865,879 

1.14 

B52G 

43,909,467 

36,073,742 

0.82 

B52H 

20,415,995 

36,107,341 

1.77 

C5A 

71,036,363 

50,403,420 

0.71 

C130B 

4,698,837 

(5,223,867) 

(1.11) 

C130E 

18,837,210 

29,406,501 

1.56 

C141B 

51,264,650 

36,080,213 

0.70 

FBlllA 

13,471,819 

26,930,368 

2.00 

As  one  might  have  guessed  from  the  preceding 
discussion,  this  model  did  not  come  through  the  validation 
test  with  flying  colors.  As  previously  identified,  the 
model  predicts  "negative"  condemnations  costs  in  some 
instances.  In  this  case,  the  FY  1981  C-130B  requirement  is 
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estimated  to  be  negative  $5.2  milliDn  when  the  actuals  were 
really  $4.7  million.  The  largest  absolute  error  (almost  $21 
million)  is  seen  in  the  FY  1981  C-5A  estimate.  Although  the 
F-4E  estimate  is  within  eleven  percent  of  the  actual,  the 
average  absolute  percentage  error  among  the  fourteen 
estimates  was  63.6  percent. 

Model  Sensitivity .  Each  model  was  also  tested 
(with  the  same  database  used  above)  to  determine  how 
sensitive  it  was  to  changes  in  the  values  of  its  independent 
variables.  The  best  linear  model  was  relatively  insensitive 
to  a  plus  or  minus  tv/enty  percent  change  in  the  "annual  MDS 
sorties"  variable.  The  average  increase  in  the  condemnation 
estimate  was  only  5.6  percent. 

Adjustments  to  the  second  variable,  "thrust  per 
engine,"  provided  mixed  results.  If  the  original 
condemnations  cost  estimate  for  an  MDS  is  small  relative  to 
the  size  of  the  model's  parameter  estimate,  variable  value, 
or  product  of  the  two,  it  is  dramatically  impacted 
(percentage  wise)  by  the  plus  or  minus  twenty  percent 
adjustment.  For  example,  the  A-7K  original  condemnation 
costs  estimate  was  only  $2.48  million.  The  "thrust" 
parameter  value,  multiplied  by  the  value  for  thrust,  was 
15.2  million.  Increasing  the  "thrust"  variable  resulted  in 
a  new  estimate  that  was  2.23  times  the  size  of  the  original 
estimate.  Over  the  entire  data  set,  the  average  estimate 
change  attributable  to  adjusting  this  variable  was  24.3 
percent . 
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Adjusting  the  third  variable  "maximum  takeoff  weight 
divided  by  maximum  empty  weight  (TOEMPWT),"  had  the  greatest 
impact  upon  the  model  estimates.  For  example,  the  original 
C-130B  estimate  was  "negative"  $5.2  million.  Increasing  the 
"TOEMPWT"  by  twenty  percent  resulted  in  a  new  estimate  of 
$11.8  million.  The  A-7K  estimate  increased  by  727  percent. 
Here  once  again  the  relative  size  of  the  parameter  value 
compared  to  the  original  estimate  made  a  difference.  The 
parameter  value  was  45.65  million  compared  to  an  original 
estimate  of  only  $2.48  million.  The  average  estimate  change 
attributable  to  adjusting  this  variable  was  145.5  percent. 

It  is  clear  from  these  results  that  the  linear  model  is 
extremely  sensitive  to  the  TOEMPWT  variable.  The  large 
parameter  values  in  this  model  led  to  greater  percentage 
changes  in  the  estimates  than  the  percentages  used  to  adjust 
the  variables.  Appendix  P  contains  the  complete  results  of 
the  sensitivity  test  for  the  linear  and  arithmetic 
transformation  models. 

Smal 1  Database  Performance .  The  final 
test  run  on  each  model  was  to  reduce  the  model  development 
database  by  replacing  annual  MDS  values  (for  those  variables 
which  changed  over  time)  with  a  single  MDS  average.  This 
test  was  performed  to  evaluate  the  impact  of  data  smoothing 
on  the  model  statistics. 

Upon  condensing  the-  best  linear  model’s  database,  most 
of  the  statistics  were  actually  degraded.  The  model's  F- 
Value  fell  from  57.013  to  12.34.  The  C.V.  rose  from  40,96 
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to  45.20.  Only  the  improved  slightly  from  12. IS  percent 
to  74.01  percent.  Additionally,  the  data  smoothing  did  not 
aide  in  the  interpretation  of  residual  plots  as  had  been 
hoped.  Given  that  this  type  of  data  smoothing  did  not 
improve  the  model  performance,  no  further  analysis  was 
performed  using  the  smaller  data  set  and  this  linear  model. 

Best  Arithmetic  Model .  In  general  the  arithmetic 
transformations  did  not  fair  much  better  than  the  linear 
models.  In  two  instances,  extremely  high  Rs  (92%+)  and 
relatively  low  C.V.s  (19%  and  24%)  were  obtained;  but  both 
of  these  models  involved  extremely  high  multicol linearity 
and  counter-intuitive  parameter  signs.  The  "best" 
arithmetic  transformation  includes  the  same  variables  as 
those  in  the  "best"  linear  model;  but  the  "maximum  takeoff 
weight  divided  by  maximum  empty  weight"  variable  is  squared 
(one  of  the  transformations  indicated  by  the  linear  model 
partial  regression  residual  plot).  The  arithmetic  equation 
for  this  CER  is  as  follows: 

CONDEMN  =  -50,924,161  +  141 . 626044 ( SORT)  +  1,140.836199 

(THRUST)  +  9,469,327(TOEMPWT2)  (8) 

where 

CONDEMN  =  annual  condemnation  requirement  ($) 

SORT  =  MDS  annual  sorties 

THRUST  =  maximum  thrust  per  engine  (lb) 

TOEMPWT2  =  (maximum  takeoff  weight  (lb)/maximum 
empty  weight  (Ib))^ 
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A  standard  quadratic  equation  was  also  attempted  which 
included  both  the  TOEMPWT  and  T0EMPWT2  terms;  but  this 
model's  performance  measurements  were  inferior  to  the  model 
using  TOEMPWT2  alone. 

The  transformation  of  TOEMPWT  to  T0EI-1PWT2  did  not  have 
a  great  impact  upon  the  model's  performance  relative  to  the 
linear  model.  Once  again,  this  model  is  capable  of 
predicting  negative  condemnation  costs  due  to  the  large, 
negative  Y-intercept.  The  absolute  value  of  the  term  is 
less  than  one  half  that  of  the  linear  model's  Y-intercept 
but  it  still  is  a  large  negative  number.  This  model's  ANOVA 
statistics  are  also  similar  to  the  linear  model's.  Table  7 
provides  the  SAS  ANOVA  table. 


Table  7 

Arithmetic  Transformation  Model  Analysis  of  Variance 


Source 

DF 

Sum  of 
Squares 

Mean 

Square  F  Value 

Prob>F 

Model 

3 

2.09961E16 

6.9987E15  58.645 

0.0001 

Error 

66 

7.8764732E15 

1.193405E14 

C  Total 

69 

2.8872573E16 

Root  MSE  10924307.8761  R-square  0.7272 
Dep  Mean  26944355.4429  Adj  R-sq  0.7148 
C.V.  40.54396 


Parameter  Estimates 


Variable 


Parameter 
DP  Estimate 


Standard  T  for  HO: 

Error  Parameter=0  Prob>lT! 


INTERCEP  1 
SORT  1 
THRUST  1 
TOEMPWT 2  1 


-50924161 

141.626044 

1140.836199 

9469327 


6306361.7189 

21.89513677 

177.95185884 

830485.53975 


-8.075 

6.468 

6.411 

11.402 


0.0001 

0.0001 

0.0001 

0.0001 
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The  good  qualities  of  this  model  mirror  those  of  the 
linear  model.  Each  of  the  cost  drivers  are  significant  to 
the  99.9  percent  level  and  they  all  possess  the  anticipated 

A 

positive  sign.  The  adjusted  R  value  is  very  close  to  the 
R  value  and  the  overall  model  s  P-Value  is  very  low 
(.0001).  This  model  is  also  parsimonious  in  its  use  of  cost 
drivers . 

Unfortunately,  this  models  shares  the  same  negative 
ANOVA  attributes  as  well.  The  is  only  negligibly  higher 
at  72.72%  (compared  to  the  linear  model's  72.16%).  The  C.V. 
value  is  also  only  slightly  improved  (40.54396  compared  to 
40.96038).  The  prediction  intervals  did  not  increase 
significantly  from  the  center  of  the  data  ($22,026,530)  to 
the  outer  edges  of  the  data  ($23,918,251);  but  like  the 
linear  model,  the  prediction  interval  was  already  so  large 
in  the  center  of  the  data  that  this  positive  attribute  is 
inconsequential . 

Figure  4  illustrates  how  this  model's  prediction 
intervals  are  impacted  by  adjustments  to  the  level  of 
confidence  used  in  the  model  development.  The  intervals 
decrease  by  almost  fifty  percent  as  one  moves  from  the  SAS 
provided  95%  confidence  level  (plus  or  minus  $22,026,530)  to 
a  70%  confidence  level  (plus  or  minus  $11,524,720).  These 
bounds,  once  again,  are  only  slightly  improved  (smaller) 
over  the  linear  model  bounds. 
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Figure  4.  Arithmetic  Transformation  Model  Confidence  Level 

Sensitivity 


Residual  Plots .  The  same  three  forms  of  residual 
plots  used  on  the  linear  model  were  applied  to  this  model 
also.  The  plots  vfere  almost  identical.  In  other  words,  the 
transformation  of  TOEMPWT  did  not  sighiftcantly  change  the 
partial  regression  residual  plot  which  had  originally 
suggested  the  need  for  a  transf oriiiation  (Figure  2).  If 
changing  TOEMPWT  to  TOEMPWT?.  had  been  the  answer,  one  would 
expect  the  new  plot  to  show  a  linear  pattern.  The  second 
alternative  to  squaring  the  TOEMPWT  variable  was  performing 
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a  log-log  transformation  on  the  entii e  model.  This  type  of 
transformation  is  discussed  in  the  n«  xt  model  subsection. 

Outliers .  Seventeen  ‘f  thi  seventy  datapoints  in 
the  model  development  databc--.-  i*  <?  ;  dent;  f ied  as  outlier 
with  respect  to  Y  by  their  iic- .-j d  residual  values. 

With  the  exception  of  one  C-5h  qatapc  int  ))eing  swapped  for  a 
C-141B  datapoint,  these  outlie.-.,  were  the  same  as  those 
identified  in  the  linear  model  analysis.  The  databases  used 
to  develop  the  two  models  are  identical,  except  for  the 
TOEMPWT  values  which  are  squared  in  this  model;  and  because 
the  number  of  variables  are  the  same,  the  cutoff  values  used 
in  the  various  outlier  tests  are  also  the  same. 

The  different  outlier  inf luentiallity  tests  provided 
mixed  results.  The  arithmetic  transformation  model's  Dffits 
test  identified  six  data  points  as  being  influential 
outliers  with  respect  to  Y.  The  T-38A  was  the  sole  addition 
to  the  linear  model's  list  of  MDS  containing  these  outliers 
(P-16A,  C-5A,  and  KC-135A) .  The  Dfbata  test  showed  that 
four  F-16A  datapoints,  one  T-38A  datapoint,  three  C-5A 
datapoints,  and  three  KC-135A  datapoints  were  influential 
outliers.  All  but  two  of  these  datapoints  were  identical  to 
the  linear  model's  results.  The  Cook's  D  test  failed  to 
identify  any  influential  outliers. 

The  outlier  problem  fo)  this  model  was  essentially  the 
same  as  that  in  the  linear  v.iodel .  As  with  all  the  models 
tested,  no  attempt  was  made  to  adjust  the  data  due  to  a  lack 
of  information  supporting  appeopriate  adjustments. 
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Multicol linearity.  Both  the  tolerance  values  and 
COLLINOINT  condition  numbers  indicated  that  there  was  no 
significant  multicol linearity  problem  in  the  model.  The 
smallest  tolerance  value  was  .665  (with  a  cutoff  value  of 
,10),  The  largest  condition  number  was  1.95  (with  a  cutoff 
value  of  ten) . 

Heteroscedasti^:it^  .  The  datapoint  residual  versus 
pred.cted  condemnaticn  value  plot  looked  exactl;  like  the 
linear  model's  corresponding  plot  (Figure  3).  ‘.nee  again, 
however,  no  action  was  taken  to  correct  for  the  apparent 
heteroscedasticity , 

Hodel  Predictive  Capability.  Table  8  addresses 
the  predictive  capability  of  this  model.  The  validation 

TABLE  8 

Arithmetic  Transformation  Model  Validation  Test  Result.^ 


ACTUAL 

BEST 

Estimate 

CONDEMN 

LINEAR 

Divided  by 

MDS 

COSTS 

ESTIMATE 

ACTUAL 

F16A 

27,537,555 

39,006,511 

1.42 

F4D 

24,151,890 

18,413,977 

0.76 

F4E 

24,254,355 

21,467,963 

0.89 

A7D 

20,075,237 

10,081,295 

0.54 

A7K 

1,560,822 

3,126,105 

2.00 

AlOA 

12,012,017 

27,935,278 

2.33 

FlllD 

14,831,043 

17,001,586 

1.15 

B52G 

43,909,467 

35,646,512 

0.81 

B.52H 

20,4X5,995 

35,708,541 

1.75 

C5A 

71,036,363 

51,640,071 

0.73 

C130B 

4,698,837 

(4,224,276) 

(0.90) 

C130E 

18,837,210 

27,056,999 

1.44 

Ci41B 

51,264,650 

35,027,409 

0.68 

PBlllA 

13,471,819 

26,312,317 

1 . 05 
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data  set  found  in  Appendix  E  was  run  through  the  model  to 
determine  how  well  this  model  performed. 

Just  as  in  the  linear  model,  the  C-IC'OB  stands  out  like 
a  sore  thumb  because  the  model  predicts  "regative" 
condemnation  requirements.  Because  only  living  creatures 
regenerate  their  broken  parts,  this  type  cf  estimate  would 
be  difficult  to  sell.  The  largest  absolu»e  error  was 
associated  with  the  C-5A  ($19.4  «tillxon).  The  average 
absolute  percentage  error  among  the  lourtten  estimates  is 
60.9  percent--a  2.7  pe’.cent  improvement  o’er  the  linear 
model . 

Model  Sensitivity.  The  ar.i.thme*  ic  transformation 
model  was  relatively  insensitive  to  the  "UDS  annual  sorties” 
variable  adjustment.  Increasing  thii  vat  .able  by  twenty 
percent  increased  the  condemnation  cost  e:;timates,  on 
average,  by  only  6.79  percent.  Adjusting  the  second 
variable,  "thrust  per  engine,"  in  the  same  way  increased  the 
condemnation  cost  estimates,  on  average,  by  25.29  percent. 

A  twenty  percent  increase  in  the  tliird  variable,  "(maximum 
takeoff  weight  divided  by  maximum  empty  weight)^"  resulted 
in  the  largest  average  estimate  incr ease--135 . 07  percent. 

These  estimate  increases  were  similar  to  the  linear 
model  results.  The  first  twe-  variable"  were  impacted 
slightly  more  in  the  arithmetic  transformation  model;  but 
the  third  variable's  average  estimate  increase  was  over  ten 
percent  less  (in  the  arithmetic  model).  Because  the  third 
variable  was  ‘he  most  sensitive  of  the  lot,  the  arithmetic 
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model  was,  overall,  less  sensitive  than  the  linear  model. 

As  seen  in  the  linear  model,  those  MDS  with  smaller  initial 
condemnations  costs  were,  percentage  wise,  impacted  more 
severely  during  the  sensitivity  test. 

Smal 1  Database  Performance.  Using  the  condensed 
database  had  practically  the  same  impact  on  this  model  as  it 
did  the  linear  model.  The  R  value  rose  slightly  from 
72.72%  to  74.63%  but  other  statistics  were  degraded.  The  F- 
Value  dropped  from  58.645  to  12.749.  The  C.V.  increased 
from  40.54396  to  44.65703.  For  these  reasons,  no  additional 
analysis  was  performed  with  the  small  database. 

Second  Best  Log-Log  Transformation  Model .  As  one  could 
see  in  the  preceding  analysis,  the  linear  and  arithmetic 
transformation  models  were  very  similar  in  their 
performance.  It  wasn't  until  log-log  transformations  were 
used  that  significant  improvements  in  the  model  statistics 
were  achieved.  It  should  be  noted  that  direct  comparisons 
between  the  two  log-log  models  presented  in  this  thesis  and 
the  preceding  models  are  hampered  by  the  fact  that  different 
databases  were  used  in  the  development  of  the  log-log 
models.  The  "MDS  annual  sorties"  and  "MDS  flyaway  cost" 
variables  were  incomplete  for  some  MDS  and  therefore, 
depending  on  which  variables  were  included  in  a  model,  the 
databases  differed.  The  first  log-log  model's  database 
excludes  the  T-37B,  T-38A,and  KC-135A  datapoints  (as  found 
in  the  first  two  models)  and  adds  FB-lllA  datapoints.  The 
second  log-log  model  discussed  also  excludes  the  same  three 
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MDS;  but  in  addition  to  the  FBlllA,  its  database  includes  P- 
15A  and  F-15B  datapoints.  The  second  log-log  model  is  also 
the  only  one  to  contain  five  datapoints  for  all  the  MDS 
contained  in  its  database  (the  other  models  contained  a  few 
MDS  with  less  than  five  datapoints).  Given  that  a  generic 
aircraft  spares  model  was  being  developed  (i.e.,  not  mission 
specific),  it  was  considered  more  important  to  include  as 
many  MDS  as  possible  in  each  model  rather  than  restricting 
the  models  to  a  common  database. 

The  mathematical  equation  for  the  second  best  log-log 
transformation  model  is  expressed  in  the  following  equation: 

CONDEMN  =  antilog(-11.621185)*(SORT)'^^’^’'*(HRSORT)-”^^-"^* 

( SPEED)  ■”^**^^*  (MAXLF)  (TOEMPWT)^'®®^^^^* 

(FLYCOST)'^^^^®^  (9) 

where 

CONDEMN  =  annual  condemnations  requirement  ($) 

SORT  =  annual  sorties 

HRSORT  =  annual  flying  hours/MDS  annual  sorties 

SPEED  =  maximum  speed  (kn) 

MAXLF  =  maximum  load  factor  (g’s) 

TOEMPWT  =  maximum  takeoff  weight  (lb) /maximum  empty 
weight  (lb) 

FLYCOST  =  flyaway  cost  ($) 

Without  going  into  the  model's  statistical  performance, 
one  improvement  in  this  model,  relative  to  its  predecessors, 
is  immediately  apparent.  Because  this  model  has  no  large 
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negative  Y-intercept  term,  it  will  always  provide  one  with 
positive  condemnation  cost  estimates. 

Looking  at  Table  9,  one  can  see  that  this  model  also 
has  many  statistical  impi  ovements  over  the  linear  and 
arithmetic  transf ormatioi.  models.  For  example,  the  R  value 
(92.81%)  for  this  model  .s  the  first  to  meet  the  acceptance 
criterion?  established  in  Chapter  III  (80.0%).  The  adjusted 
value  (92.08%)  is  very  close  to  the  R^  value,  indicating 

Table  9 

Second  Best  Log- Log  Transformation  Model 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Prob>F 

Model 

6 

56.84300 

9.47383 

126.985 

0.0001 

Error 

59 

4.40173 

0.07461 

C  Total 

65 

61.24473 

Root  MSE 

0,27314  R 

:-square 

0.9281 

Dep  Mean 

16.67773  Adj  R-sq 

0.9208 

C, 

■  V. 

1.63775 

Parameter 

Estimates 

Parameter 

Standard 

T  for  ho: 

Variable 

DF 

Estimate 

Error 

Parameter’=0 

?rob> It! 

INTERCEP 

1 

-11.621185 

1.72285885 

-6.745 

0.0001 

LSORT 

1 

0.85797'? 

0.0438854 

19.550 

0.0001 

LHRSORT 

1 

0.97684’ 

0.19575726 

4.990 

0.0001 

L SPEED 

1 

0.557096 

0.08992126 

6.195 

0.0001 

LMAXLF 

1 

0.525784 

0.14409791 

3.64 

0.0006 

LTOEMPWT 

1 

3.085454 

0.53437101 

5.74 

0.0001 

LFLYCOST 

1 

0.696182 

0.09304748 

7.82 

0.0001 

indicating  that  the  increase  in  R^  is  not  due  siroply  to  an 
influx  of  insignificant  independent  variables.  in  fact,  all 
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of  the  individual  variables  are  significcnt  to  the  99.9% 
level  and  the  F-Value  (126.985)  more  than  doubles  the 
previous  models'  statistics.  Additionally,  the  overall 
model's  P-Value  is  very  low  (.0001). 

The  model's  Root  MSE  value  (27.514%'  also  represents  a 
significant  improvement;  but  it  stil  .  isn't  as  low  as  one 
would  like  to  see.  A  figure  closer  o  ten  percent  would 
have  provided  a  much  warmer  feeling  bout  this  model's 
predictive  accuracy. 

The  prediction  interval  varied  ignificantly  depending 
upon  whether  one  was  near  the  center  of  the  data  or  not. 
Using  the  SAS  95%  prediction  interva  bounds,  it  was  noted 
that  at  the  center  of  data  the  width  of  the  prediction 
interval  (converted  to  arithmetic  sp- ce)  was  11,324,214.  At 
the  outer  edge  of  the  data,  the  pred  ction  interval  width 
grew  to  57 ,356,284--a  difference  of  6,032,070,  Although 
this  model  is  more  sensitive  to  a  da'apoint's  distance  from 
the  center  of  the  data,  one  must  remember  that  the  previous 
models  had  poor  intervals  even  in  th  ;  cent*-r  of  the  data. 
Being  consistently  bad  is  not  an  imp  ovement.  Only  when  one 

moves  to  the  extreme  high  end  of  the  data  does  the  log-log 

model  have  larger  prediction  interva  widths. 

One  must  be  careful  when  compar  ng  prediction  interval 
performance  for  a  log-log  model  with  that  of  a  linear  or 

arithmetic  transformation  model ,  Be  ng  at  the  center  of  the 

data  in  a  log-log  model  does  not  guarantee  that  the 
arithmetic  width  of  the  prediction  interval  is  the  smallest 
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in  the  model  (as  it  does  in  the  linear  and  arithmetic 
transformation  models).  This  datapoint's  prediction 
interval,  when  converted  to  arithmetic  space,  is  the 
smallest,  percentage  wise,  relative  to  the  predicted  value 
of  the  dependent  variable  (Y).  As  will  be  seen  in  the 
second  log-log  model,  if  the  center  of  the  data  is  located 
nearer  the  larger  predicted  values  for  Y,  the  prediction 
interval  can  be  rather  large  and  still  be,  percentage  wise, 
the  best  predictor  of  Y. 

The  only  area  in  which  this  log-log  model  is  weaker 
than  its  predecessors  is  in  its  "ease  of  application."  This 
model  has  more  cost  drivers  and  therefore  additional  data 
must  be  gathered  before  it  can  be  used.  However,  the 
variables  were  considered  logical  and  they  all  tested  to  be 
significant.  If  one  were  to  exclude  significant  variables 
simply  for  ease  of  application,  one  would  have  to  accept  a 
lower  R^. 

The  next  test  conducted  was  the  confidence  level 
sensitivity  test.  The  center  of  the  data  was  located  in 
logarithmic  space  and  then  converted  to  arithmetic  space. 

Figure  5  shows  how  the  bounds  (converted  to  arithmetic 
space)  are  impacted  by  varying  the  level  of  confidence.  As 
in  the  previous  models,  moving  the  level  of  confidence  from 
70%  to  95%  had  a  tremendous  impact  on  the  prediction 
interval  width.  The  width  more  than  doubles  from  5,698,179  :i 

to  11,324,210.  I 
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Figure  5.  #2  Log-Log  Transformation  Model 

Confidence  Level  Sensitivity 


Residual  Plots .  The  same  three  forms  of  residual  | 
plots  used  in  the  previous  analysis  were  also  used  on  the  ^  J 
log-log  models.  Once  again,  all  of  the  standard  residual  I 
plots  appeared  to  show  randomly  scattered  datapoints.  For  J 
this  model,  however,  neither  of  the  two  additional  residual  | 
plot  forms  identified  the  need  for  a  variable  '4 

A 

transformation.  The  patterns  in  thei.e  plots  appeared  to  be  -j 
positively  sloped  and  linear,  indicating  that  the  correct  I 
transformations  had  been  found.  It  should  be  noted  that  the  1 
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data  dispersion  was  greater  for  the  "TOEMPWT"  and  "MAXLF" 
variables . 

Outliers .  Using  a  log-log  transformation  did  not 
improve  the  outlier  problem  seen  in  the  previous  models. 
Fifteen  of  the  66  datapoints  used  in  this  model  were 
identified  as  outliers  with  respect  to  Y  by  their 
studentized  residual  values.  The  MDS  represented  in  these 
outliers  included  the  F-4D,  F-4E,  A-7D,  A-7K,  B-52G,  B-52H, 
C-130B,  C-141B,  and  FB-lllA.  The  leverage  values  for  only 
one  of  the  66  datapoints  indicated  it  was  an  outlier  with 
respect  to  X.  This  was  a  C-5A  datapoint.  The  percentage  of 
outliers  actually  increased  with  this  transformation. 

The  number  of  outliers  which  were  influential  was  also 
high  according  to  the  Dfbeta  test.  Fifteen  different 
datapoints  were  influential  outliers  for  one  variable  or 
another  according  to  this  test.  The  MDS  included  in  this 
category  were  the  F-4D,  F-4E,  A-7D,  A-7K,  A-lOA,  C-130B,  C- 
130E,  C-141B,  and  FB-lllA.  The  Dfits  test,  however, 
indicated  that  only  four  datapoints  (including  F-4E,  C-130B, 
and  A-7K  datapoints)  were  influential  outliers  with  respect 
to  Y.  The  Cook's  D  test,  however,  revealed  no  influential 
outliers.  Once  again,  no  action  was  taken  to  alleviate  the 
outlier  condition. 

Multicol linearity.  Whether  or  not  one  believes 
there  is  a  serious  multicol linearity  problem  in  this  model 
depends  upon  which  test's  rule  of  thumb  one  has  more  faith 
in.  The  rule  of  thumb  used  for  the  tolerance  test  is  that 


each  variables 's  tolerance  value  must  be  ten  percent  or 
higher.  The  "HRSORT"  variable  has  a  tolerance  value  of 
.07621283.  The  rule  of  thumb  used  in  this  thesis  for  the 
COLLINOINT  test  is  that  the  highest  condition  number  can  be 
equal  to  or  greater  than  ten.  This  model’s  largest 
condition  number  is  8.19456.  Even  if  a  large  degree  of 
multicol linearity  exists,  it  doesn't  necessarily  mean  that 
the  model  will  not  continue  to  predict  in  a  consistent 
manner.  If  the  multicollinearity  is  constant  over  time, 
such  as  when  it  is  due  to  some  physical  law,  the  model  will 
continue  to  predict  as  well  as  if  the  multicollinearity  was 
not  present.  According  to  the  COLLINOINT  test,  the 
multicollinearity  relationship  appears  to  be  between  the 
"flying  hours  per  sorty,"  "maximum  load  factor,"  "ratio  of 
maximum  takeoff  weight  to  maximum  empty  weight,"  and 
"flyaway  cost"  variables.  Given  the  time  constraints  of 
this  thesis,  there  was  inadequate  time  to  thoroughly  study 
the  relationship  among  these  variables  to  determine  if  a 
constant  relationship  should  be  expected.  Because  the  final 
log-log  model  (discussed  in  the  next  model  subsection)  did 
not  appear  to  have  a  severe  multicollinearity,  no  effort  was 
made  to  adjust  this  model.  Given  the  mixed  results  of  the 
tests,  and  the  possibility  that  the  multicollinearity  is 
constant  over  time,  it  hasn't  really  been  determined  that 
there  is  a  need  for  any  adjustment. 

Heteroscedasticitv .  The  "datapoint  residual 
versus  predicted  condemnations  plot"  did  not  suggest  the 
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presence  of  heteroscedasticity  in  this  model .  The  scatter 
pattern  maintained  a  fairly  constant  width  across  the  data. 

Model  Predictive  Capabi litv .  Table  10  shows  how 
well  this  log-log  transformation  nodel  performed  in 
predicting  costs  for  the  validation  data  set  (see  Appendix 
E) .  Each  estimate  produced  by  SAS  was  multiplied  by  a  bias 
adjustment  factor  of  L.038.  The  average  absolute  percent 
error  among  the  fourteen  estimates  was  30.6  percent.  This 
is  a  considerable  improvement  over  the  previous  models' 
percent  error  (63.6  perce''>t  and  60.9  percent)  and  as  was 
already  stated,  this  model  provides  no  "negative"  cost 

TABLE  .1.0 

#2  Log-Log  Transformation  Model  Validation  Test  Results 


ACTUf  L 

#2 

Estimate 

CONDEl-  N 

Log-Log 

Divided  by 

MDS 

COSTS 

ESTIMATE 

ACTUAL 

F16A 

27,537 , 555 

39,235,008 

1.42 

F4D 

24,151,890 

14,723,360 

0 . 61 

F4E 

24,254,355 

21,934,295 

0.90 

A7D 

20,075,237 

9,165,037 

0.46 

A7K 

1,560,822 

1,690,062 

1.08 

AlOA 

12,012,017 

25,926,341 

2.16 

FlllD 

14,831,043 

13,946,359 

0 . 94 

B52G 

43,909,467 

39,742,968 

0 . 91 

B52H 

20,415.995 

25,424,376 

1.25 

C5A 

71,036,363 

45,137,070 

0 . 64 

C130B 

4,698  837 

2,881,839 

C  .  61 

C130E 

18,837  210 

15,197,728 

0.81 

C141B 

51,264.650 

59,692,585 

1.16 

FBlllA 

13,471,819 

12,312,288 

0 . 91 
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estimates.  However,  the  largest  absolute  error  was  $25.9 
million  (for  the  C-5A),  compared  to  $21  million  and  $19.4 
loi  l.he  previous  models. 

Model  Sensitivity.  Table  11  presents  the  results 
of  the  model  sensitivity  test.  For  each  cost  drave-r,  the 
impact  of  a  plus  or  minus  twenty  percent  change  in  X  (the 
cost  driver  value)  is  presented.  Ar  one  can  see,  the  model 
IS  relatively  insensitive  to  changer  in  ail  the  cost  drivers 
except  one  "the  ratio  of  maximum  La] eoff  weight  to  maximum 
empty  weight"  (TOEMPWT) .  A  twenty  i ercent  increase  in  this 
variable  results  in  a  76  percent  in  ;rease  in  the  cost 
estimate.  The  only  other  variable  .■jhich,  when,  adjusted  by 
twenty  percent,  caused  a  percent  adjustment  in  the 


Table  11 

#2  Log-Log  Transformation  Model 
Sensitivity  Test 


Delta  Y  due  to  Delta  Y  due  to 

Variable  increase  in  X  decrease  in  X 


SORT 

+  .17 

-.29 

HRSORT 

+  .19 

-.20 

SPEED 

+  .11 

-.12 

MAXLF 

^  .10 

-.11 

TOEMPWT 

+  .76 

-.50 

FLYCOST 

+  .14 

-  .14 

condemnations  estimate  greater  than  twenty  percent  was  the 
"annual  sorties"  variable.  When  the  SORT  value  was 
multiplied  by  .8,  the  cost  estimate  decreased  by  29  percent. 
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Smal 1  Database  Performance .  The  condensed 
database  test  resulted  in  mixed  results.  The  model's  R‘ 
went  up  from  92.81  percent  to  97.04  percent  and  the  C.V. 
fell  from  27.314  percent  to  25.285  percent.  These  welcomed 
results  were  offset  by  the  fact  that  the  model's  F-Value 
fell  from  126.985  to  38.314.  Additionally,  the  condition 
value  on  the  COLLINOINT  test  went  up  to  9.8942.  For  these 
reasons,  no  further  analysis  was  performed  on  the  condensed 
data  set  model . 

Best  Loo-Log  Transformation  Model .  The  "best"  log-log 
transformation  model  was  actually  outperformed  by  the  log- 
log  model  discussed  above  in  several  respects,  but  it  was 
decided  that  its  overall  performance  made  this  last  model 
the  best  one  developed  during  the  analysis. 

The  mathematical  equation  for  the  "best"  log-log 
transformation  model  is  expressed  in  the  following  equation: 

CONDEMN  =  antilog(-9.185307)*(ACNUM)^-®^^’^^*(THRUST)-^’^^^* 

( TOEMPWT )  ^  *  (  FLY  COST  )  ’  ( 10  ) 

where 

CONDEMN  =  annual  condemnations  requirement  ($) 

ACNUM  =  annual  number  of  aircraft  in  MDS  inventory 

THRUST  =  maximum  thrust  per  engine  (lb) 

TOEMPWT  =  maximum  takeoff  weight  (lb)/maximum  empty 
weight  (lb) 

FLYCOST  =  flyaway  cos  ($) 
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Like  the  first  log-log  model  just  deicribed,  this  model 
always  provides  positive  estimates  (  .e.,  the  sign  is 
positive).  Since  the  superiority  of  the  last  log-log  model 
over  the  linear  and  arithmetic  transj ormation  models  was 
made  clear  in  the  last  model  subsection,  the  following  text 
will  focus  on  the  comparison  of  the  two  log-log  models. 

Table  12  provides  the  ANOVA  statistics  for  the  "best" 
log-log  transformation  model.  As  pr^viou-sly  stated,  there 
are  several  statistics  in  which  the  'irst  log-log  model 
performs  better  than  this  model.  Fo.  example,  the  first 
log-log  model's  R'  value  (92.81%)  is  slightly  better  than 

Table  12 

Best  Log-Log  Transformation  Model 
Analysis  of  Variance 


Sum  of 

Mea  . 

Source 

DF 

Squares 

Squan* 

F  Value 

Prob>F 

Model 

4 

78.44007 

19.61012 

239.074 

0.0001 

Error 

75 

6.15187 

0.082  )2 

C  Total 

79 

84.59193 

Root 

MSE 

0.28640 

R-square 

0.9273 

Dep 

Mean 

16.64209 

Adj  R-sq 

0.9234 

C.V. 

1.72094 

Parameter 

Es:imates 

Parameter 

Standard 

T  for  HO: 

Variable 

DF 

Estimate 

Error 

Parameter=0 

Prob> 1 T ! 

INTERCEP 

1 

-9.185307 

1.14373918 

-8.031 

0.0001 

LACNUM 

1 

1.018916 

0.05170451 

19.707 

0.0001 

LTHRUST 

1 

0.743450 

0.11291517 

6.584 

0.0001 

LTOEMPWT 

1 

2.491597 

0.44931709 

5.545 

0.0001 

LFLYCOST 

1 

0.667079 

0.08913384 

7.484 

0.0001 
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that  of  this  model  (92.73%).  Both  model's  adjusted  R^s  are 
close  to  their  R^,  and  so  the  overall  difference  in  the  R^ 
statistic  is  minuscule.  The  first  model's  Root  MSB 
(27.314%)  is  also  slightly  better  than  that  of  this  model 
(28.640%).  The  individuals  variables  in  both  models  are 
highly  significant  (99.9%)  and  they  both  have  very  small  P- 
Values  ( . 0001) . 

The  first  major  difference  in  the  models  is  in  their  F- 
Values.  The  "best"  model's  F-Value  (239.074)  almost  doubles 
that  of  the  first  log-log  model  (126.985),  This  indicates 
that  it  is  a  more  statistically  significant  model.  The  best 
log-log  model  also  has  two  fewer  "f-st  drivers.  This  will 
make  the  data  gathering  process  easier  for  this  model . 

The  prediction  interval  analysis  for  the  best  log-log 
model  was  interesting  in  that  the  bounds  widths  behaved 
differently.  Like  the  last  model,  the  prediction  interval 
varied  significantly  depending  upon  whether  one  was  near  the 
center  of  the  data  or  not.  Using  the  SAS  95%  prediction 
interval  bounds,  one  can  see  that  at  the  center  of  data  the 
width  of  the  prediction  interval  (converted  to  arithmetic 
space)  is  50,932,443.  Though  this  interval  is  significantly 
wider  than  the  first  model's  (11,324,212),  this  doesn't  mean 
that  the  model  will  always  have  larger  bounds.  At  the  outer 
edge  of  the  data  (where  X  is  the  smallest),  the  prediction 
interval  actually  shrinks  to  a  width  of  4,142,632--a 
reduction  of  46,789,811.  The  fact  that  this  model's  bounds 
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are  wider  in  the  center  of  the  data  simply  means  that  the 
point  at  which  this  mcde).  predicts  the  best,  percentage 
wise,  occurs  at  higher  estimated  values  of  Y.  The  magnitude 
of  the  prediction  interval  change  is  actually  only  758 
thousand  higher  in  this  model . 

Figure  6  displays  the  confidence  level  sensitivity  test 
results.  Like  the  other  models,  the  results  show  that  the 


-w-  Pr^dicfed  Vdite  Lower  Bound  Uppoi  Bouid 


Figure  6.  Best  Log-Log  Transformation  Model  Confidence 
Level  Sensitivity 


prediction  interval  almost  doubles  as  the  level  of 
confidence  increases  from  70  percent  to  95  percent.  In 


order  to  show  the  relative  size  of  the  various  model's 
bounds  when  the  same  scale  is  used  for  all  the  models,  one 
additional  graph  (Figure  7)  was  created.  The  bounds  for  the 
linear  and  arithmetic  transformation  models  were  essentially 
the  same  width  and  therefore,  to  make  a  cleaner  graph. 

Figure  7  includes  only  the  linear  bounds.  The  two  log-log 


model  bounds  are  also  plotted.  While  the  widths  of  the  best 
log-log  model  and  the  linear  model  appear  to  be 
approximately  the  same,  the  intervals  for  the  log-log  model 
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occur  at  a  much  higher  predicted  val  le  of  Y.  A  plus  or  minus 
$12.8  million  bound  for  an  estimate  )f  $41.6  million  is  much 
more  accurate  (percentage  wise)  than  a  plus  or  minus  $11.6 
million  bound  for  an  estimate  of  $19.9  million. 

Residual  Plots .  The  same  :hree  forms  of  residual 
plots  used  in  the  previous  analysis  were  also  used  on  this 
log-log  models.  The  model  performed  essentially  the  same  as 
first  log-log  model.  Once  again,  al  of  the  standard 
residual  plots  appeared  to  show  randtmly  scattered 
datapoints.  Neitherthe  of  the  two  auditional  residual  plot 
forms  identified  the  need  for  a  vari.-.ble  transformation. 

The  data  dispersion  was  a  little  tigliter  for  this  model 
however. 

Outliers.  The  best  log-lot;  transformation  still 
possessed  an  outlier  problem.  Seventeen  of  the  eighty 
datapoints  used  in  this  model  were  identified  as  outliers 
with  respect  to  Y  by  their  studentizf.d  residual  values.  The 
MDS  represented  in  these  outliers  included  the  F-15A,  F-15B, 
P-4D,  F-4E,  A-7D,  A-7K,  B-52G,  B-52H,  C-5A,  C-130B,  C-130E, 
and  FB-lllA.  None  of  the  datapoints  were  outliers  with 
respect  to  X  according  to  the  leverage  values. 

Once  again,  the  number  of  outliers  which  were 
influential  was  high  according  to  the  Dfbeta  test.  Fourteen 
different  datapoints  were  influentia  out’iers  for  one 
variable  or  another,  including  F-15A  F-15B,  F-4E,  A-7K,  C- 

5A,  C-130B,  and  FB-lllA  datapoints.  The  Jfits  test 
indicated  that  only  four  datapoints  including  one  each  of 
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the  F-4E,  A-7K,  C-5A,  and  C-130B  datapoints)  were 
influential  Dutliers  with  respect  to  Y.  Once  again,  no 
action  was  taken  to  alle/iate  the  outlier  condition. 

MulticolljnearLtv.  Neither  the  tolerance  values 
or  the  COLLINOINT  ccndition  values  indicated  the  presence  of 
significant  multico} linearity  in  this  model.  The  highest 
condition  number  was-  only  4.3115,  compared  to  8.19456  for 
the  last  model.  Tht.*  lowest  tolerance  value  was  .19756894. 
This  is  a  significant  reduction  in  the  level  of 
multicol linearity  within  the  model. 

Heteroscedasticitv.  The  "datapoint  residual 
versus  predicted  condemnations  plot"  did  not  suggest  the 
presence  of  heteroscedas licity  in  this  model.  The  scatter 
pattern  maintained  a  fai.ly  constant  width  across  the  data. 

Model  Predict!  e  Capability.  Table  13  shows  how 
well  this  log-log  transformation  model  performed  in 
predicting  costs  for  the  validation  data  set  (see  Appendix 
E) .  The  SAS  provided  esLimates  were  multiplied  by  a  bias 
adjustment  factor  of  1.042  to  arrive  at  the  estimates  in 
Table  13.  The  average  absolute  percent  error  among  the 
fourteen  estimates  was  27  percent,  compared  to  30.6  percent 
for  the  first  log-log  model.  The  model  provides  no 
"negative"  cost  estimates  and  the  largest  percentage  error 
was  77  percent  (for  the  A-lOA),  compared  to  116%  for  the 
same  datapoint  in  the  first  log-log  model.  The  largest 
absolute  error  was  $13,477,655  (for  the  C-5A)  compared  to 
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$25,89''. 293  for  the  same  datapoint  ii  the  first  log-log 
model . 


TABLE  13 

Best  Log-Log  Transformation  Model  V..lidatLon  Test  Results 


ACTUAL 

Be;  t 

Estimate 

CONDEMN 

Log-  Log 

Divided  by 

MDS 

COSTS 

ESTIhATE 

ACTUAL 

F16A 

27,537,555 

46,493,354 

1.68 

F4D 

24,151,890 

18,820,898 

0.78 

F4E 

24,254,355 

24,178,775 

1.00 

A7D 

20,075,237 

12,534,068 

0.62 

A7K 

1,560,822 

1,518,977 

0.97 

AlOA 

12,012,017 

21,269,329 

1.77 

FlllD 

14,831,043 

11,966,556 

0.81 

B52G 

43,909,467 

42,115,435 

0.96 

B52H 

20,415,995 

27,415,449 

1.34 

C5A 

71,036,363 

57,558,708 

0.81 

C130B 

4,698,837 

3,149,006 

0.67 

C130E 

18,837,210 

12,688,688 

0.67 

C141B 

51,264,650 

43,537,881 

0.85 

FBlllA 

13,471,819 

11,701,858 

0.87 

Model  Sensitivity.  Table  14  presents  the  results 

of  the  model  sensitivity  test.  For  jach  cost  driver,  the 

Table  14 

Best  Log-Log  Transformation  Model 

Sensitivity  Test 

Delta  Y  due  to 

Delta  Y  due 

to 

Variable 

increase  in  >. 

decrease  in 

X 

ACNUM 

+  .20 

-.20 

THRUST 

+  .15 

-.15 

TOEMPWT 

+  .58 

-.43 

FLYCOST 

+  .13 

-.14 
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impact  of  a  plus  or  minus  twenty  percent  change  in  X  (the 
cost  driver  value)  is  presented. 

Like  the  first  log-log  model,  this  model  is  relatively 
insensitive  to  changes  in  all  the  cost  drivers  except  one 
"the  ratio  of  maximum  takeoff  weight  to  maximum  empty 
weight"  (TOEMPWT).  A  twenty  percent  increase  in  this 
variable  results  in  a  58  percent  increase  in  the  cost 
estimate  (compared  to  76  percent  for  the  first  log-log 
model ) . 

Smal 1  Database  Performance .  The  condensed 
database  test  provided  mixed  results  for  this  model.  Using 
this  database,  the  model's  went  up  from  92.73  percent  to 
95.36  percent.  Its  Root  MSE  dropped  from  28.64  percent  to 
26.38  percent.  Reducing  the  database  size  resulted  in  a 
large  drop  in  the  model's  F-Value.  It  fell  from  239.074  to 
56.555.  The  small  improvements  were  not  considered 
significant  enough  to  pursue  further  analysis  using  the 
small  database. 

Demand  Volatility  Analysis 

This  section  is  dividea  into  two  subsections,  the  first 
section  presenting  the  results  of  a  replenishment  spares  CER 
and  the  second  section  looking  at  spreadsheet  generated 
demand  volatility  factors. 

Replenishment  Spares  CER.  The  ultimate  goal  of 
producing  the  condemnation  CERs  described  in  the  preceding 
text  was  to  take  the  estimates  generated  by  these  models  and 
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use  them  to  predict  annual  replenishment  spares  costs.  As 
discussed  in  Chapter  II,  this  type  o.‘:  approach  is  currently 
included  in  the  HQ  AFLC/FMC's  Logistics  Support  Cost  Model 
(LSC)  [the  LSC  model,  however,  is  an  accounting  type  model 
as  opposed  to  a  CER] .  An  attempt  wa.s  made  to  improve  upon 
the  accuracy  of  the  demand  volatility  factor  used  in  the  LSC 
model  to  convert  condemnation  costs  to  replenishment  spares 
costs.  The  following  text  presents  the  results  of  this 
effort . 

A  SIV  replenishment  spares  model  was  created  with  SAS 
using  annual  MD  condemnation  costs  as  the  sole  cost  driver. 
Appendix  G  provides  the  condemnations  cost  database  and 
replenishment  spares  requirements  database  used  in 
development  of  the  SIV.  The  tables  in  this  appendix  also 
include  the  mission  design  (MD)  and  fiscal  year  associated 
with  each  datapoint.  The  arithmetic  expression  of  this 
model  is  expressed  in  the  following  equation: 

REPLEN  =  -10,613,130  +  3 . 75853 (CONDEMN)  (11) 

where 

REPLEN  =  annual  replenishment  spares  requirement  ($) 
CONDEMN  =  annual  condemnations  cost  ($) 

This  model  has  the  same  problem  found  in  the  linear  and 
arithmetic  transformation  condemnation  CERs--it  has  a  large 
negative  intercept  and  therefore  can  result  in  negative  cost 
estimates.  This  model  also  has  poor  statistics,  as  one  can 
see  in  the  ANOVA  table  provided  below. 


9 
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The  "condemnations  cost"  variable  has  the  correct 
parameter  estimate  sign  and  is  a  significant  cost  driver 
(P-Value  of  .0001).  The  model's  F-Value  (129.849)  is  also 
greater  than  that  seen  in  the  first  two  models;  but 
everything  else  goes  down  hill  from  there.  The  value 
(59.06)  suggests  that  there  are  other  sources  of  error  not 
captured  in  this  model.  This  statistic  is  well  below  the  80 
percent  acceptance  criterion  stated  at  the  end  of  Chapter 
III.  The  C.V.  (58.46779)  tells  one  that  this  model  is  not  a 
very  reliable  predicter  of  annual  replenishment  spares 
requirements.  This  is  confirmed  in  Figure  8,  which  presents 
the  SAS  95%  prediction  intervals  for  this  model.  The 
interval  width  is  extremely  large  and  still  some  datapoints 
fall  outside  the  bounds. 


Table  15 

Replenishment  Spares  CER  Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

P  Value 

Prob>F 

Model 

1 

1.6559149E18 

1.6559149E18 

129.849 

0.0001 

Error 

90 

1.1477359E18 

1.2752621E16 

C  Total 

91 

2.8036508E18 

Root  MSE  112927504.960  R-square  0.5906 
Dep  Mean  193344805.913  Adj  R-sq  0.5861 
C.V.  58.46779 

Parameter  Estimates 


Variable  DP 


Parameter  Standard  T  for  HO: 

Estimate  Error  Parameter=0  Prob>lT! 


INTERCEP  1  -10613130  21409126.648  -0.496  0.6213 

CONDEMN  1  3.758530  0.32983652  11.395  0.0001 
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The  model's  database  also  has  an  abundance  of  outliers, 
as  seen  in  the  preceding  models.  Twelve  out  of  92 
datapoints  are  identified  as  being  outliers  with  respect  to 
Y  and  sixpoints  are  outliers  with  respect  to  X.  Of  these 
numbers,  eight  datapoints  were  tested  (Dffits  test)  as  being 
influential  outliers  with  respect  to  Y.  Six  of  these  eight 
datapoints  and  one  additional  one  were  also  identified  as 
being  influential  outliers  by  the  Dfbetas  test.  The  Cook's 
D  test  did  not  identify  any  influential  outliers.  Once 
again,  no  action  was  taken  to  adjust  the  database. 

Although  the  statistics  of  this  model  are  not  very 
impressive,  they  do  provide  one  with  a  sense  of  how  accurate 
one  can  expect  to  be  in  using  the  model.  The  analysis 
supporting  the  demand  volatility  factor  used  in  the  LSC 
model  did  not  provide  s  ich  statistics. 

An  additional  SAS  nodel  was  created  with  the  Y- 
intercept  restricted  to  a  value  of  zero.  This  action 
flattened  the  slope  f ou  id  in  the  previous  model  so  that  one 
doesn't  obtain  "negativ  i"  estimates.  The  slope  for  this 
model  (3.621965)  is  the  value  one  would  use  as  the  demand 
volatility  factor.  The  ANOVA  statistical  measurements  are 
no  longer  meaningful  f o  :  this  model  because  of  the 
restriction  placed  upon  the  model.  The  demand  volatility 
factor  from  this  model,  3.621965,  is  significantly  larger 
than  the  2.15  value  used  in  the  LSC  model. 

Demand  Volati lity  Analysis .  In  addition  to  the  SAS 
generated  SIV  model  described  above,  demand  volatility 


i 
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factors  were  developed  by  comparing  JID  annual  condemnation 
costs  with  the  annual  replenishment  spares  requirements 
developed  for  these  MDs  two  years  earlier.  Table  16 
provides  the  results  of  this  analysis.  The  missing  demand 
volatility  factors  in  the  T-37  and  T-38  data  sets  are  due  to 
holes  in  the  replenishment  spares  requirements  data  set. 

The  one  missing  demand  volatility  factor  in  the  FB-111  data 
set  is  due  to  the  fact  that  this  MD  is  being  converted  to 
another  F-111  MDS  (Rosenthal,  1991).  The  factor  was 
abnormally  high  and  it  was  felt  that  this  was  due  to  the 
start  of  the  conversion  process. 

In  addition  to  the  individual  demand  volatility 
factors,  averages  are  provided  both  by  FY  and  within  the 
MDs.  In  fact,  two  different  averages  are  provided.  The 
first  is  the  average  of  all  the  individual  demand  volatility 
factors  by  category  (FY  and  MD) .  The  second  average  is 
based  upon  the  raw  data  behind  the  demand  volatility  factors 
(i.e.,  rather  than  just  averaging  the  factors,  the  total 
replenishment  spares  requirements  are  divided  by  the  total 
condemnation  costs).  The  first  average  give  equal  weight  to 
each  of  the  individual  factors,  reg;  rdless  of  the  MD's 
relative  contribution  to  total  cost: .  The  raw  data  average 
gives  more  weight  to  high  cost  itemr  . 

Additionally,  Table  16  includes  the  results  of  a  simple 
t' end  analysis.  If  the  last  three  datapoints  within  a 
particular  MD  data  set  were  either  lower  or  higher  than  the 
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Demand  Volatility  Analysis 
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data  set  average,  a  downward  (DWN)  or  upward  (UP)  trend  was 
identified . 

Although  numerous  individual  demand  volatility  factors 
were  less  than  the  factor  (2.15)  ustd  in  the  LSC  model,  only 
the  T-37  (with  only  three  factors  ii.  the  data  set)  had  an  MD 
average  which  was  less.  In  fact,  m^.ny  of  the  factor 
averages  were  substancial ly  higher  than  2.15.  Looking  at 
the  FY  averages  shows  that  none  were  less  than  the  current 
LSC  factor.  The  average  demand  volatility  factor  for  all 
MDs  across  all  the  PYs  was  3.46.  The  raw  data  average  was 
just  a  little  higher--3 . 56 .  Again,  this  value  is 
substancial ly  higher  than  the  current  LSC  factor. 

When  one  looks  at  the  trends  in  the  MD  factor  data 
sets,  however,  one  sees  that  half  of  the  MDs  have  downward 
trends  and  none  exhibit  upward  trends.  The  FY  averages 
confirm  this.  The  annual  average  demand  volatility  factors 
steadily  decrease  each  year.  The  annual  raw  data  averages 
do  not  exhibit  a  steady  decrease,  but  the  trend  is 
definitely  downward.  The  2.15  factor  used  in  the  LSC  model 
comes  much  closer  to  the  averages  experienced  over  the  last 
few  FYs. 

The  overall  downward  trend  in  the  factor  averages  is 
consistent  with  the  tightening  of  DoD  budget  purse  strings 
seen  in  recent  years.  The  first  few  years  of  the  analysis 
database  included  the  "Reagan  build-up  years."  A  part  of 
Ronald  Reagan's  campaign  strategy  had  bei^n  to  suggest  that, 
under  President  Carter,  the  DoD  had  become  a  hollow  force. 
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ill-equiped  to  adequately  protect  the  nation.  Funding  was 
not  an  issue  in  these  years  and  therefore  the  replenishment 
spares  requirements  were  much  higher.  The  abundance  of 
funding,  however,  was  not  sustained  across  the  entire 
analysis  database.  The  lower  factor  averages  in  the  latter 
years  of  the  analysis  may  be  due,  at  least  in  part,  to  this 
fact . 

Besides  looking  foi  treads  in  the  factor  averages,  the 
individual  factors  were  also  evaluated  to  see  if  they 
appeared  to  fall  into  groupings  by  aircraft  mission  (e.g., 
fighter,  bomber,  etc.).  There  was  too  much  variability 
within  the  different  mission  categories  and  insufiicient 
sample  size  to  draw  conclusive  results.  It  was  interesting 
to  note,  however,  that  some  of  the  high  cost  MDs ’  averages 
were  consistently  highei  than  the  current  LSC  factor.  The 
F-16  and  F-15  factors,  in  particular,  have  consistently  been 
closer  to  four  than  to  2.15, 

While  no  mission-unique  demand  volatility  factors  are 
suggested  from  this  analysis,  it  is  clear  that  a  single 
factor,  used  over  a  number  of  years  for  all  aircraft,  will 
introduce  a  significant  amount  of  error  to  one's  estimate. 


V..„  Summary  and  Recommendations 


lJ5  troduction 

This  Chapter  summarizes  the  results  of  the  annual 
condemnations  cost  model  development  and  demand  volatility 
analysis.  Recommendations  for  use  of  the  models  developed 
and  for  further  study  are  also  provided. 

Summary 

The  major  objective  of  this  thesis  was  to  develop  a 
parametric  model  for  estimating  annval  replenishment  spares 
costs  based  upon  aircraft  physical  and  performance 
characteristics.  The  methodology  for  achieving  this 
objective  was  divided  into  two  parts.  First,  a  CER  for 
estimating  annual  condemnation  costs  was  developed  and  then 
the  relationship  between  condemnation  costs  and 
replenishment  spares  requirements  was  evaluated.  The 
analysis  of  this  relationship  was  further  divided  into  two 
steps:  1)  a  SIV  model  for  estimating  replenishment  spares 
costs  was  developed  using  annual  condemnations  as  the  sole 
cost  driver;  and  2)  annual  replenishment  spares  requirements 
(by  MD)  were  compared  to  annual  condemnation  costs  to  arrive 
at  factors  known  as  "demand  volatility”  or  "churn"  factors. 

Four  condemnations  CERs  were  evaluated  in  the  text. 

The  best  linear  and  arithmetic  transformation  models  failed 
to  produce  statistically  acceptable  results.  The  two  best 
log-log  transformation  models  provided  significantly 
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improved  results;  but  sill  their  predictive  accuracy  is 
questionable.  While  th*  study  showed  that  performance  and 
physical  characteristic;  could  be  used  to  produce 
statistically  significai  t  models,  there  was  too  much 
variability  in  the  data  (as  evidenced  by  the  number  of 
outliers)  to  achieve  de;  irable  model  prediction  intervals. 

The  demand  volatil  ty  factor  analysis  suggests  that  one 
factor  (for  all  aircraft  and  all  fiscal  years)  does  not 
adequately  represent  the  changing  conditions  which  impact 
the  relationship  of  rep! enishment  spares  requirements  to 
condemnation  costs.  A  c  ownward  trend  in  the  factor  averages 
was  observed  across  the  iata--indicating  the  need  for 
continuing  trend  analys;  s  to  update  any  factor  used.  No 
mission-unique  factors  \  ere  suggested  due  to  variability  in 
the  factors  and  inadequ  te  mission-unique  sample  sizes. 

Recommendations  f o  Use.  The  first  two  models  analyzed 
for  this  thesis  (linear  and  arithmetic  transformation 
models)  are  not  recomme.  ded  for  use  due  to  their  poor 
statistical  performance  The  two  log-log  transformation 
models  are  viable  cost  i  odels  for  predicting  annual 
condemnation  costs.  Be  ause  neither  model  was  blessed  with 
small  prediction  interv  Is,  however,  it  is  recommended  that 
they  are  used  for  compa  ison  purposes  with  other  estimating 
approaches  and  not  as  tl  e  primary  estimating  tool , 
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The  replenishment  spares  CER  hal  even  poorer  statistics 
than  the  linear  and  arithmetic  trans ;ormation  condemnation 
CERs  and  therefore  isn't  recommended  for  use. 

The  overall  demand  volatility  a/erage  was  significantly 
higher  than  that  currently  used  by  HQ  AFLC/FMC  in  their 
Logistics  Support  Cost  Model.  However,  there  is  a  downward 
trend  in  the  data  and  if  one  were  to  average  the  factors 
over  the  last  three  years  in  the  data  set,  the  factors  would 
not  be  nearly  as  far  apart.  Given  the  number  of  adjustments 
made  to  the  replenishment  spares  database  so  that  it  could 
be  compared  with  the  condemnation  ccsts  catabase  (see 
Chapter  III),  it  would  probably  onli  be  vanity  that  suggests 
this  thesis's  factor  average  is  superior  to  the  one 
currently  in  use.  This  point  is  really  mute,  however, 
because  the  analysis  suggests  that  cne  factor  should  not  be 
used  for  all  aircraft  and  across  numerous  fiscal  years.  The 
Mb  factor  averages  are  recommended  as  stfirting  points  for 
developing  analogous  weapon  system  demand  volatility 
factors.  Because  trends  were  identified  in  the  data,  future 
users  should  first  update  the  database  with  the  latest  data 
available  before  using  the  MD  factor  averages. 

Recommendations  for  Future  Study .  There  are  several 
areas  related  to  this  research  which  would  benefit  from 
additional  study.  The  whole  topic  of  demand  volatility  is  a 
relatively  unexplored  domain  with  very  little  research 
conducted  and  even  less  written  down.  Though  one  would 
think  that  condemnations  should  be  the  major  driver  of 
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replenishment  spares  requirements,  what  little  research  that 
has  been  conducted  to  date  indicates  otherwise. 

If  the  number  of  replenishment  spares  required  each 
year  truly  exceeds  the  condemnations  by  the  ratios  of  two  to 
one  and  greater,  one  must  wonder  what  happens  to  all  the 
spares.  With  aircraft  modifications  occurring  continuously, 
perhaps  a  large  number  o:  spares  become  obsolete.  Whether 
data  exists  for  this  kini  of  study  is  unknown;  but  the 
research  would  be  enlightening. 

This  demand  volatility  factor  analysis  examined  whether 
or  not  the  factors  tended  to  group  by  mission  category. 
Additional  study  should  lie  conducted  to  determine  if  the 
work  unit  code  unique  factors  could  be  developed.  For 
example,  avionics  spares  may  have  a  different  factor  average 
than  the  airframe  spares. 

If  a  suitable  database  is  discovered  or  developed,  the 
same  research  conducted  in  this  thesis  for  replenishment 
spares  can  be  conducted  cor  initial  spares. 


Appendix  A;  Definition  of  Terms 


Unless  specified  otherwise,  the  following  definitions 
are  taken  from  the  AFLC  Cost  Analysis  Handbook  (TASC,1989): 


Additives  -  Initial  spare  "additives"  are  special  stock 
level  requirements  over  and  above  those  determined  by 
the  standard  AFLC  Form  614  process.  Replenishment 
spares  additives  are,  as  the  name  implies,  requirements 
which  are  calculated  separately  from  the  D041 
calculations  and  added  manually  into  the  D041  system 
(Rosenthal,  1991). 

Analogy  Based  Estimating  -  method  of  cost  estimating 
based  upon  adjusting  the  actual  cost  of  a  similar 
existing  system  for  "complexity,  technical,  or  physical 
differences,"  between  it  and  the  new  system  for  which 
the  estimate  is  being  derived  (Ch  14,  27). 

Availability  -  A  measure  of  the  degree  to  which  an  item 
is  in  an  operable  and  committable  state  at  the  start 
of  a  mission  when  the  mission  is  called  for  at  an 
unknown  (random)  time.  (Ch  5,  31) 

Base  Repair  Cycle  Time  (BRCT)  Pipeline  -  The  average 
number  of  spares  resident  in  the  base  level  repair 
process  at  any  given  point  in  time  (Dement,  1990;Appen 
1,  sec  2.1.2)  . 

Condemnations  -  When  the  cost  to  repair  a  spare  exceeds 
75%  of  the  cost  to  purchase  a  new  spare,  the  item  is 
"condemned"  (Novak,  1991).  Condemnation  spares, 
therefore,  are  those  spares  purchased  to  replace  the 
condemned  spares. 

Cost  Estimating  Relationship  (CER)  -  a  mathematical 
relationship  that  relates  one  variable,  usually  cost 
(called  the  dependent  variable),  to  one  or  more  other 
cost  drivers  (called  the  independent  variables).  (Ch 
10,  3) 

Cost  Drivers  -  "Those  [independent]  variables  that 
exhibit  some  systematic  relationship  with  cost"  (Ch  10, 
3). 

Demand  Based  Sparing  -  A  category  of  inventory 
control/cost  estimating  models  which  base  their 
stockage  recommendations  upon  the  probability  of 
meeting  the  demand  for  a  spare  part  out  of  available 
stock  (AFLCP  57-13,  1987:10). 
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Demand  Volatility  -  Condenuiation  spares  are  only  a 
subset  of  the  total  replenishment  spares  requirement. 
The  difference  between  these  two  requirements  is 
attributed  to  "demand  volatility"  (as  known  as 
"churn").  The  causes  of  demand  volatility  include:  1) 
replenishment  spares  funds  are  used  to  purchase 
pipeline  spares  when  initial  spares  funds  ar<» 
inadequate,  2)  changes  flying  hour  program.  parts 
utilization  rates  and  mair.tenance  factors,  and  3) 
additive  requirements  (No\ak,  1991). 

Depot  Repair  Cycle  Time  (DRCT)  Pipeline  -  The  average 
number  of  spares  in  the  depot  level  repair  process 
which  were  originally  generated  at  the  base  level 
(Dement,  1990:Appen  1,  sec  2.2.1). 

Engineering  Cost  Estimate  -  An  approach  to  cost  estima 
ting  that  "encompasses  a  detailed  'build-up'  of  labor 
hours,  material  costs,  and  overhaul  at  very  finite  sub 
indentures  of  the  program/activity/item  for  which  cost 
is  to  be  estimated"  (Ch  14,  28). 

Flyaway  Costs  -  Non-recurring  plus  recurring  costs  for 
airframe,  propulsion  and  avionics,  program  management, 
test  and  evaluation,  [and]  allowances  for  engineering 
changes.  (Levine  and  Horowitz,  1989:5) 

Initial  Spares  -  Repairable  components  which  support 
newly  fielded  end  items  (or  principal  items)  for  the 
entire  production  run  of  the  aircraft  (Rexroad,  Tillia 
and  Tritle,  1990:21). 

Independent  Cost  Estimate  (ICE)  -  A  cost  estimate  per¬ 
formed  by  an  "honest  broker"  with  no  ties  to  the 
program  office.  It  is  used  as  a  check  for  the 
reasonableness  of  the  program  cost  estimate  (Ch  14,  8) 

Initial  Spare  Support  List  (ISSL)  -  ISSLs  include  thos 
provisioned  items  which  the  equipment  specialist  and 
the  using  conmiand  agree  should  be  stocked  and  on-hand 
for  support  of  operations  and  maintenance  at  the 
operational  location  when  the  first  items  become 
operational.  (Reynolds,  1989:15) 

Job  Routed  (JR)/Non-JR  Repair  Cycle  Time  Pipeline  and 
Depot  Overhaul  Stock  -  spares  required  to  cover 
"programmed  depot  maintenance  and  component  overhaul" 
(Dement,  1990:Appen  1,  sec  2.2.2). 

Life  Cycle  Cost  (LCC)  -  The  total  cost  to  the 
Government  of  acquisition  and  ownership  of  the  system 
over  its  full  life.  It  includes  the  cost  of 


development,  acquisition,  operation,  support,  and  where 
applicable,  disposal.  (Ch  5,  33) 

Line  Replaceable  Unit  (LRU)  -  Ar  aircraft  is  composed 
of  major  subassemblies  such  as  the  avionics  system. 
These  systems  are,  in  turn,  composed  of  final 
assemblies  or  components  known  es  LRUs  (AFLCP  57-13, 
1987:3) . 

Maintainability  -  A  system  or  component  characteristic 
which  "refers  to  the  ease  with  vhich  a  given 
component/system  can  be  maintain ed"  (Ch  5,  18). 

Mean  Time  Between  Demand  (MTBD)  -  A  derivative  of  the 
mean  time  between  failure  (MTBF)  which  has  been 
modified  to  account  for  demand  c rivers  not  included  in 
the  MTBF.  For  example,  the  MTBI  does  not  include 
component  failures  which  are  bo yond  the  contractor's 
responsibility  such  as  operator  error  and  mishandling 
(Reynolds,  1989:81-82). 

Mean  Time  Between  Failure  (MTBF)  -  For  a  particular 
interval,  the  total  functional  life  of  a  population  of 
an  item  divided  by  the  total  number  of  failures  within 
the  population.  (Ch  5,  33) 

Most  Probable  Cost  (MPC)  -  An  estimate  used  during 
source  selection  to  check  the  reasonableness  of  a 
bidder's  cost  proposals  (Ch  14,  9). 

Obligation  -  As  used  in  the  Air  Force  program  control 
community,  funds  are  said  to  be  "obligated"  at  that 
point  in  time  when  the  contractual  agreement  between 
the  Air  Force  and  contractor  is  posted  into  the 
official  accounting  and  finance  records.  "Obligations" 
are  separate  from  "commitments"  and  "expenditures." 

The  former  takes  place  when  a  purchase  request  or  other 
authorized  commitment  document  is  signed  by  the 
accounting  &  finance  certifying  official  and  the  latter 
takes  place  when  the  Air  Force  pays  the  Contractor. 

Operations  and  Support  (OSS)  Costs  -  Fixed  and  variable 
costs  of  personnel,  material,  facilities,  and  other 
items  needed  largely  for  the  peacetime  operation, 
maintenance,  and  support  of  a  system  during  activation, 
steady  state  operation,  and  disposal.  (Ch  17,  3) 

Order  and  Shipping  Time  (OST)  Pipeline  -  The  number  of 
spares  required  to  cover  the  time  to  order  and  ship 
spares  to  the  base  from  the  depot  whenever  components 
are  sent  to  the  depot  for  repair  or  are  condemned  [see 
"condemnations"  above]  at  the  base.  It  is  based  on  the 
average  shipping  and  handling  times  and  average  supply 


demand  rates  at  the  base  level.  (Dement,  1990: Append 
1,  sec  2.1.1) 

Procurement  Lead-time  -  The  sum  of  the  administrative 
and  production  lead  times.  Administrative  lead  time 
(ALT)  is  "the  period  of  time,  in  whole  months,  from  the 
start  of  III  [item  manager]  preparation  of  purchase 
request/military  interdepartmental  purchase  request 
(PR/MIPR)  to  date  of  contract  award"  (AFLCR  57-4, 
1983:Ch  1,  5).  Production  lead  time  (PLT)  is  "the  time 
between  award  of  the  contract  or  purchase  order  and 
resultant  first  significant  delivery  quantity"  (AFLCR 
57-4,  1983:Ch  1,  6). 

Procurement  Lead-time  Spares  -  spares  which  are 
purchased  to  ensure  adequate  stock  during  the 
administrative  and  production  lead-times  for  items" 
(Dement,  1990: Append  1,  sec  2.3.2). 

Program  Cost  Estimate  (PCE)  -  The  "program  manager's 
official  estimate  of  the  financial  resources  required 
to  competently  conduct  the  program  contained  in  its 
Program  Management  Directive  (PMD)"  (Ch  14,  8). 

Provisioning  -  The  process  of  determining  and  acquiring 
the  range  and  quantity  (depth)  of  spares  and  repair 
parts,  and  support  and  test  equipment  required  to 
operate  and  maintain  an  end  item  of  material  for  an 
initial  period  of  sei'vice.  (Ch  5,  34) 

Provisioning  Technical  Documentation  -  A  "generic 
term  for  all  (he  provisioning  data  developed  by  the 
contractor,  including  listings,  drawings,  diagrams, 
schematics,  etc."  (Reynolds,  1989:10). 

Recoverable  Spares  -  "Repairable  parts,  assemblies, 
components,  etc.  used  in  the  repair  of  higher  level 
assemblies"  (Reynolds,  1989:49). 

Readiness  Based  Sparing  -  A  category  of  inventory  con¬ 
trol/cost  estimating  models  which  base  their  stockage 
recommendations  upon  maintaining  sufficient  spares  to 
ensure  aircraft  availability  goals  are  met.  The  key 
difference  between  this  approach  and  demand  based 
sparing  is  that  the  demand  for  a  spare  does  not  neces¬ 
sarily  imply  that  an  aircraft  cannot  perform  its 
mission  (Horner,  1991). 

Reliability  -  The  probability  that  the  system  w.xll 
satisfy  the  need  for  which  it  was  intended  in  an 
acceptable  manner,  for  a  given  period  of  time,  when 
deployed  and  used  under  a  given  set  of  operating 
conditions.  .  .  .  Satisfactory  performance  describes 
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the  level  at  which  the  item/system  must  perform; 
performance  below  this  level  is  then  considered 
"failure"  even  though  the  specific  part/ component  has 
not  broken  or  reached  a  zero  performance  level .  (Ch 
5,  16) 

Repair  Parts  -  "Consumable  non-repairabl e  parts  used  to 
repair  higher  level  assemblies"  (Reynolds,  1989:49). 

Replenishment  Spares  -  Repairable  components, 
assemblies,  or  subassemblies  required  to  resupply 
initial  stockage  or  increased  stockage  for  reasons 
other  than  support  of  newly  fielded  end  items. 
Replenishment  would  include  additional  stockage  due  to 
increases  such  as  usage,  readiness  initiatives,  and 
redeployment  of  end  items.  (Ch  17,  40) 

Safety  Level  -  The  quantity  of  spares  held  at  both  the 
base  and  depot  repair  facilities  to  account  for 
fluctuations  in  the  pipeline  requirements  (Dement, 
1990:Appen  1,  sec  2.1.3  S  2.2.3). 

Shop  Replaceable  Unit  (SRU)  -  The  subcomponents  which 
together  compose  a  line  replaceable  unit  (LRU)  are 
known  as  SRUs  (AFLCP  57-13,  1987:3). 

Should  Cost  Estimate  (SCE)  -  A  cost  estimate  developed 
for  individual  contracts  as  a  basis  for  the 
Government's  negotiation  objective  (Ch  14,  9). 

Single  Best  Estimate  (SBE)  -  A  cost  estimate  which 
results  when  the  independent  cost  estimate  and  program 
cost  estimate  processes  are  combined  as  one.  Both 
program  office  and  staff  cost  analysts  combine  to  form 
a  joint  estimating  team  in  developing  the  SBE  (Ch  14, 
9). 

Source,  Maintenance,  and  Recoverability  (SMR)  Code  -  A 
code  assigned  to  every  item  (both  recoverable  spares 
and  repair  parts)  identified  in  the  provisioning 
technical  documentation.  The  code  "indicates  the 
method  of  support  [i.e.,  base,  intermediate,  or  depot 
repair],  authorized  maintenance  actions  [e.g.,  repair, 
reconditioning,  etc.],  and  appropriate  disposal 
authority  for  each  item"  (Reynolds,  1989:12-13). 
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Appendix  B;  Recoverable  Spares  Cost  Estimating  Background 


Introduction 

The  purpose  of  this  appendix  is  to  provide  additional 
background  information  for  those  readers  new  to  the  field  of 
e.ircraft  recoverable  spares  cost  estimating.  The  text 
begins  with  definitions  for  key  terms.  Following  this  are 
descriptions  of  the  current  methodologies  for  developing 
both  spares  budgets  and  buy  requirements.  These 
descriptions  are  divided  into  separate  subsections  devoted 
to  the  two  major  classes  of  recoverable  spares--initial  and 
replenishment — beginning  with  the  former  class.  After  this, 
a  discussion  of  how  spares  cost  estimating  relates  to  the 
life  cycle  phase  of  the  aircraft  is  provided.  Related 
subtopics  in  this  section  are:  1)  the  maintenance  plan 
development,  2)  the  various  methodologies  for  cost 
estimating,  and  3)  the  different  reasons  for  cost  estimates. 
Finally,  the  appendix  concludes  with  an  analysis  of  the 
different  sources  of  data  relating  to  recoverable  spares 
evaluated  for  this  thesis. 

Key  Terms 

Before  proceeding  any  further  in  the  text  it  is 
important  that  several  key  terms  be  defined.  First,  the  two 
major  classes  of  recoverable  spares  (those  which  can  be 
repaired  when  broken)  are  defined  as  follows: 
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Initial  Spares  -  Repairable  components  which  support 
newly  fielded  end  items  (or  principal  items)  for  the 
entire  production  run  of  the  aircraft.  The  budget  will 
support  stockage  at  all  levels  including  the  pipeline 
Initial  spares  will  also  include  whole  spare  engines. 
(Rexroad,  Tillia,  and  Tritle,  1990:21) 

Replenishment  Spares  -  Repairable  components, 
assemblies,  or  subassemblies  required  to  resupply 
initial  stockage  or  increased  stockage  for  reasons 
other  than  support  of  newly  fielded  end  items. 
Replenishment  would  include  additional  stockage  due  to 
increases  such  as  usage,  readiness  initiatives,  and 
redeployment  of  end  items.  (TASC,  1989:Ch  17,  40) 

These  definitions  requires  some  clarification  and 

further  definition.  Before  1985,  initial  spares  would  only 

cover  newly  fielded  aircraft  for  an  average  of  two  years. 

After  this  point,  all  spares  were  considered  replenishment 

spares,  even  if  the  actual  production  line  for  the  new 

aircraft  continued  on  for  many  years.  Under  the  new 

definition  of  initial  spares,  each  production  lot  is  covered 

for  a  period  of  approximately  two  years  before  transitioning 

to  replenishment  spares.  If  the  production  line  provides 

(for  example)  ten  lots  of  aircraft  over  a  ten  year  period, 

each  lot  is  covered  by  initial  spares  for  its  own  two  year 

period  (Rexroad,  Tillia,  and  Tritle,  1990:21).  There  were 

aircraft  whose  production  line  began  before  the  new 

definition  and  continued  on  afterv/ards.  For  these  aircraft, 

if  the  two  year  initial  spares  coverage  period  had  already 

ended  and  replenishment  spares  were  being  used  when  the 

definition  changed,  they  continued  to  use  replenishment 

spares  funds  to  cover  all  future  spares  requirements.  In 

other  words,  these  aircraft  were  "grandfathered"  under  the 
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old  definition.  Those  aircraft  still  in  the  midst  of  their 
two  year  initial  spares  coverage  adopted  the  new  definition 
and  thus  future  production  lots  received  their  own  initial 
spares  coverage  period  (Rosenthal,  1991;  Neuhart,  1991). 

The  initial  spares  definition  mentions  that  this  class 
covers  the  "pipeline".  Because  recoverable  spares  can,  by 
definition,  be  repaired  there  are  always  a  number  of  spares 
in  the  process  of  being  repaired  or  in  transit  to  or  from 
the  repair  facilities.  The  "pipeline"  is  divided  into 
several  pieces.  The  Oiler  and  Shipping  Time  (OST)  Pipeline 
is 

the  time  to  order  .nd  ship  spares  to  the  base  from  the 
depot  whenever  com  >onents  are  sent  to  the  depot  for 
repair  or  are  cond jmned  [see  "condemnation  spares" 
below]  at  the  base.  It  is  based  on  the  average 
shipping  and  handling  times  and  average  supply  demand 
rates  at  the  base  level.  (Dement,  1990;Append  1,  sec 
2.1.1) 

The  Base  Repair  Cycle  T.  me  (BRCT)  Pipeline  is  "the  average 
number  of  spares  resident  in  the  base  level  repair  process 
at  any  given  point  in  time"  (Dement,  1990:Appen  1,  sec 
2.1.2).  The  Depot  Repa .r  Cycle  Time  (DRCT)  Pipeline  is  "the 
average  number  of  spares  in  the  depot  level  repair  process 
which  were  originally  generated  at  the  base  level"  (Dement, 
1990:Appen  1,  sec  2.2.1).  The  Job  Routed  (JR)/Non-JR  Repair 
Cycle  Time  Pipeline  and  Depot  Overhaul  Stock  provide  spares 
to  cover  "programmed  depot  maintenance  and  component 
overhaul"  (Dement,  1990:Appen  1,  sec  2.2.2).  An  analogy  for 
this  last  subset  of  initial  spares  is  when  an  auto  mechanic 
is  performing  a  scheduled  maintenance  check  on  one's  car  and 
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he  recommends  that  a  few  parts  be  replaced.  These  parts 
have  not  yet  failed;  but  it  isn't  worth  trying  to  prolong 
their  life  when  their  failure  may  result  in  greater, 
costlier  damage  to  the  vehicle. 

also  included  in  initial  spares  requirements  are  safety 
level  stock,  procurement  lead-time,  additives,  and 
condemnation  spares.  The  base  and  depot  safety  levels 
provide  additional  spares  to  account  for  fluctuations  in  the 
pipeline  requirements  (Dement,  1990:Appen  1,  sec  2.1.3  S 
2.2.3).  AFLCR  57-27,  Initial  Requirements  Determination, 
does  not  provide  for  initial  spares  "safety  stock"  by  name 
as  seen  in  the  replenishment  spares  regulations.  It  does, 
however,  allrw  budgeting  for  the  purchase  of  initial  spares 
to  cover  the  lead-time  (the  period  of  time  from  obligation 
of  funds  to  spares  delivery)  plus  an  additional  three  month 
supply  (AFLC  57-27,  1986:60;  Rosenthal,  1991).  This 
standard  three  month  period  is  not  as  sophisticated  as  the 
marginal  analysis  techniques  used  to  develop  replenishment 
spares  safety  level  (Rosenthal,  1991)  but  it  serves  the  same 
purpose.  "Procurement  lead-time  spares  ar-i  purchased  to 
ensure  adequate  stock  during  the  administrative  and 
production  lead-times  for  items"  (Dement,  1990:Append  1,  sec 
2.3.2).  Initial  spare  "additives"  are  special  stock  level 
requirements  over  and  above  those  determined  by  the  standard 
AFLC  Form  614  process.  When  the  cost  to  repair  a  spare 
exceeds  75%  of  the  cost  to  purchase  a  new  spare,  the  item  is 
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"condemned"  (Novak,  1991).  Condemnation  spares,  therefore, 
are  those  spares  purchased  to  replace  the  condemned  spares. 

The  definition  for  replenishment  spares  implies  that 
t.his  class  of  spares  coverage  basically  picks  up  when 
initial  spares  coverage  ends.  In  reality,  the  break  between 
the  categories  is  not  as  clean  as  the  definitions  imply. 

The  term  "replenishment"  suggests  maintaining  a  previously 
established  level  of  spares.  Condemnation  spares  do  fulfill 
this  function  as  one  of  the  major  elements  of  replenishment 
spares;  but  they  are  only  a  part  of  the  total  replenishment 
spares  requirement.  There  have  been  (and  continue  to  be) 
situations  where  the  initial  si^ares  estimates  were 
inadequate  and  pipeline  requirements  were  filled  with 
replenishment  spares  funding  (Rosenthal,  1991).  The  final 
major  element  of  replenishment  spares  are  additives.  The 
"Recoverable  Consumptior.  Item  Requirements  System"  (D041) 
computes  replenishment  spares  requirements  based  upon 
historical  demand.  Replenishment  spares  additives  are,  as 
the  name  implies,  requij ements  which  are  calculated 
separately  from  the  D04;.  calculations  and  added  manually 
3.nto  the  D041  system  (Rosenthal,  1991).  AFLCR  57-4,  the 
governing  regulation  foi-  D041,  provides  a  list  of  the 
numerous  additive  requirements.  Replenishment  spares  also 
cover  smaller  categories  of  requirements  such  as  safety 
level  and  procurement  l«:ad-time  spares. 

Finally,  several  additional  spares  requirements  such  as 
negotiated  base  stock  levels,  forward  supply  support  levels. 
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war  readiness  material  kits,  depot  iloating  stock, 
insurance,  and  foreign  military  sal  ss  are  included  in  the 
total  recoverable  spares  requiremen..  These  categories  of 
spares  represent  special  requiremen -.s  that  are  tracked 
separately.  As  such,  they  are  not  ncluded  in  the  scope  of 
this  discussion.  One  may  look  to  A ’LCR  57-4,  Recoverable 
Consumption  Item  Requirements  Svste  t  (0041)  for  additional 
details  on  these  requirements. 

Additional  terms  requiring  def  .nition  at  this  time 
include : 

Availability  -  A  measure  of  th-j  degree  to  which  an  item 
is  in  an  operable  and  committajle  state  at  the  start  of 
a  mission  when  the  mission  is  called  for  at  an  unknown 
(random)  time.  (TASC,  1989:Ch  5,  31) 

Flyaway  Costs  -  Non-recurring  olus  recurring  costs  for 
airframe,  propulsion  and  avionics,  program  management, 
test  and  evaluation,  [and]  allowances  for  engineering 
changes.  (Levine  and  Horowitz,  1989:5) 

Provisioning  -  The  process  of  determining  and  acquiring 
the  range  and  quantity  (depth)  of  spares  and  repair 
parts,  and  support  and  test  eq\  ipment  required  to 
operate  and  maintain  an  end  it-  m  of  material  for  an 
initial  period  of  service,  (T;.SC,  1989:Ch  5,  34) 

Initial  Spares  Budget  Development 

Historically,  the  agency  responsible  for  initial  spares 
budget  preparation,  HQ  APLC/FMBSR,  has  used  a  factor  based 
approach  to  developing  their  initial  spares  requirements 
(with  the  exception  of  whole  engine  spares).  This  has 
primarily  involved  multiplying  airciaft  flyaway  cost  by  a 
spares  "factor"  [additional  factors  are  applied  against 
training  and  peculiar  support  equiprient].  These  factors  are 
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sometimes  adjusted  based  on  the  expert  opinion  of  the  budget 
manager  given  any  input  from  other  sources  such  as  the  Air 
Logistics  Centers  (ALCs)  or  System  Program  Offices  (SPOs) 
(Neuhart,  1991;  Rexroad,  Tillia,  and  Tritle,  1990:1).  The 
underlying  logic  of  this  methodology  is  limited  to  the 
assumption  that  the  greater  a  weapon  system's  flyaway  cost, 
the  more  expensive  its  spares  will  be.  It  continues  to  be 
used  because:  1)  data  inputs  required  for  other  models  are 
either  not  available  or  very  hard  to  come  by,  and  2)  it  is 
easy  to  apply. 

There  are  two  changes  to  this  historical  perspective. 
First,  HQ  AFLC/FMBSR  has  given  most  of  its  responsibility 
for  budget  preparation  to  the  ALCs  [they  still  budget  for 
common  support  equipment  and  whole  spares  engines  are 
handled  separately].  The  FY  92/93  Budget  Estimate 
Submission  (BES),  dated  September  1990,  was  the  first  ALC 
input  (Neuhart,  1991).  Second,  the  factor  based  approach  to 
developing  initial  spares  budgetary  requirements  has  been 
criticized  for  its  lack  of  insight  into  the  underlying 
causal  relationships  driving  the  estimates  (Dement,  1990: Sec 
1;  Rexroad,  Tillia,  and  Tritle,  1990:1)  and  efforts  are 
underway  to  develop  a  new  methodology. 

For  a  while,  demam'  based  approaches  similar  to  those 
used  in  the  initial  spates  provisioning  process  were 
considered  the  likeliest  candidates  to  replace  the  factor 
based  approach.  This  mentality  is  seen  in  a  coordinated 
message  from  SAF/FMC,  SAF/AQK,  SAF/AQX,  and  HQ  USAF/LEX, 
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dated  2  October  1990,  vihich  stated  that  the  Air  Force  Cost 
Analysis  Improvement  Group  (AFCAIG)  preferred  the  use  of 
"valid,  demand  based  models  for  calculating  initial  and 
condemnation  spares  for  POE's  [Program  Office  Estimates]  and 
ICA's  [Independent  Cost  Estimates]  presented  to  them  for 
Defense  Acquisition  Board  (DAB)  milestones"  (SAF/FMC,  1990, 
2) .  Demand  based  approaches  such  as  the  Logistics  Support 
Cost  (LSC),  and  ModMETRIC  (see  Appendix  C)  are  concerned 
with  the  probability  of  meeting  the  demand  for  a  spare  part 
out  of  available  stock  (Alexander,  1990:111-24;  AFLCP  57-13, 
1987:10).  Studies  were  conducted  to  evaluate  demand  based 
approaches  that  would  fulfill  the  initial  spares  budget 
development  role  (Dement,  1990;  Re:4road,  Tillia,  and  Tritle, 
1990);  but  before  an  "approved"  methodology  was  arrived  at, 
the  emphasis  switched  from  "demand  based"  to  "availability" 
or  "readiness  based"  approaches. 

The  use  of  readiness  based  sparing  (RBS)  methodologies 
is  expected  to  reduce  recoverable  spates  costs  (Beckett,  1). 
While  demand  based  models  try  to  ensure  that  spares  are 
available  upon  demand,  the  RBS  models  recognize  that  demand 
for  a  spare  doesn't  necessarily  imply  that  an  aircraft  must 
be  grounded  or  can’t  perform  its  mission.  The  RBS 
methodology,  rather  than  focusing  on  the  demand  for 
individual  spare  parts,  is  concerned  with  maintaining 
sufficient  spares  stock  to  ensure  aircraft  availability 
goals  are  met  (Horner,  1991).  This  theoretically  smaller 
stock  results  in  reduced  recoverable  spares  costs. 
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According  to  a  HQ  USAP/LEYS  letter  dated  14  November 
1990,  the  Office  of  the  Assistant  Secretary  of  Defense, 
Production  and  Logistics  (OASD/P&L)  was  preparing  a  new 
regulation  to  replace  DoDI  4140.42,  Determination  of 
Requirements  for  Secondary  Item  Spare  and  Repair  Parts 
Through  the  Demand  Period ,  which  will  mandate  the  use  of  RBS 
methods  (Beckett,  1).  HQ  AFLC/XRI  is  currently  evaluating 
the  RBS  methodology  used  by  the  Army  and  Navy  for  potential 
Air  Force  implementation  (Robinson,  1991). 

Until  a  practical  alternative  methodology  is  developed 
to  replace  the  factor  based  approach,  it  will  probably 
continue  to  be  relied  upon  for  developing  initial  spares 
budgets  (as  evidenced  by  the  number  of  ALCs  using  it  for 
their  FY  92/93  BES  inputs  (Neuhart,  1991)). 

Initial  Spares  Provisioning  Requirements  Development 

Due  to  the  length  of  the  Program  Objective  Memorandum 
(POM)  process  (the  long  range  budget  plan),  initial  spares 
budget  wedges  are  typically  put  in  place  several  years 
before  initial  spares  provisioning  actually  occurs  (Neuhart, 
1991).  Details  of  an  aircraft’s  maintenance  plan  required 
for  initial  spares  provisioning  are  not  always  available 
when  these  budgets  are  being  developed  (TASC,  1989 :Ch  5, 

55);  therefore,  the  factor  based  approach  to  budgeting 
discussed  above  bears  little  resemblance  to  the  method  used 
for  initial  spares  provisioning.  While  budgets  are 
submitted  for  "aircraft"  requirements,  provisioning  takes 
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place  at  the  individual  "parts"  leve' .  This  section  will 
provide  an  overview  of  the  method  usi*d  to  ietermine  the 
"range  and  depth"  of  initial  spares  i equirements  known  as 
the  initial  spares  provisioning  (Reynolds,  1989:5).  It 
should  be  remembered  that  "provisioning",  as  defined 
previously,  encompasses  more  than  initial  spares;  but  the 
scope  of  this  discussion  excludes  information  not  pertinent 
to  spares.  Unless  indicated  otherwise,  references  in  this 
section  will  come  from  a  1989  analysis  of  the  Initial  Spares 
Support  List  (ISSL)  process  sponsored  by  the  Air  Force 
Logistics  Management  Center.  The  analysis  team  was  led  by 
Captain  Steve  Reynolds.  This  reference  is  recommended  for 
readers  who  are  interested  in  an  in  depth  analysis  of  the 
entire  provisioning  process. 

Provisioning  Methods .  Activities  which  take  place 
during  provisioning  include 

the  assigning  of  Source,  Maintenance,  and 
Recoverability  codes;  assignment  and  review  of  the 
various  provisioning  factors  which  quantify  projected 
usage  requirements;  assignment  of  Item  Management 
codes;  assignment  of  Federal  Supply  Classes;  etc."  (7). 

Before  explaining  these  activities  in  more  detail,  the  three 
methods  used  for  provisioning  will  be  described  in  the 
following  order:  1)  the  Provisioning  Conference;  2)  the 
Resident  Provisioning  Team;  and  3)  the  Depot  Committee. 

The  Provisioning  Conference  is  the  most  commonly  used 
provisioning  method.  Participants  may  include  the  prime 
contractor,  the  Systems  Program  Manager  (SPM)/End  Article 
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Item  Manager  (EAIM),  representatives  from  the  using  command 
and  the  Defense  Logistics  Agency  (DLA),  and  the  Air 
Logistics  Center  equipment  specialists.  This  team  meets  on 
a  temporu..j^  at  a  time  and  place  which  is  mutually 

agreed  upon  to  accomplish  the  provisioning  function  (7). 

Major  weapon  systems  may  make  use  of  the  Resident 
Provisioning  Team  approach.  In  this  method,  the 
provisioning  team  is  a  small  group  of  specially  qualified 
personnel  who  are  assigned  on  a  permanent  basis  to  the 
contractor's  facility.  The  objective  of  this  approach  is 

to  reduce  the  time  required  to  furnish  the  contractor 
with  spare  and  repair  parts  orders,  to  achieve  a 
greater  degree  of  understanding  and  cooperation  between 
the  contractor  and  the  Air  Force,  and  to  ensure  a 
greater  degree  of  compliance  with  provisioning 
requirements  by  the  contractor.  (9) 

The  final  provisioning  approach  is  reserved  for  those 
cases  where  the  "system  or  end  item  is  not  overly  complex  or 
the  number  of  items  involved  is  not  too  great"  (10).  This 
smaller  scale  version  of  the  Provisioning  Conference  takes 
place  at  a  Depot  and  is  called  the  Depot  Committee. 

Provisioning  Technical  Documentation  ( PTD) .  PTD  is  a 
"generic  term  for  all  the  provisioning  data  developed  by  the 
contractor,  including  listings,  drawings,  diagrams, 
schematics,  etc."  (10).  The  PTD  is  the  basis  for  the 
provisioning  team's  decisions  regarding  the  range  and  depth 
of  required  spares.  The  System  Program  Office  (SPO)  is  the 
agency  responsible  for  including  PTD  requirements  in  their 
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systems  acquisition  contract.  PTD  is  entered  into  and 
maintained  on  the  AFLC  Provisioning  System  (D220)  (10). 

Provisioning  Activities .  As  previously  stated, 
provisioning  activities  include  assigning  codes  and  factors 
which  "identify  (1)  which  items  will  be  required  for  systems 
support,  (2)  how  many  items  will  be  required,  and  (3)  how 
those  items  will  be  managed  in  the  inventory"  (12). 

A  Source,  Maintenance,  and  Recoverability  (SMR)  code  is 
assigned  to  every  item  (both  recoverable  spares  and 
consumable  repair  parts)  identified  in  the  PTD.  This  code 
"indicates  the  method  of  support  [i.s.,  base,  intermediate, 
or  depot  repair],  authorized  maintenance  actions  fe.g., 
repair,  reconditioning,  etc.],  and  appropriate  disposal 
authority  for  each  item"  (12-13).  It  is  this  code,  assigned 
by  the  ALC  equipment  specialist  (12),  which  fulfills  the 
function  of  identifying  the  range  of  recoverable  spares 
required . 

Other  codes  include  the  Item  Management  Code  (IMC) 
which  assigns  management  responsibility  for  each  item  and 
the  Material  Management  Aggregation  Code  (M1<IAC)  which  groups 
related  hardware  for  management  purposes  that  might 
otherwise  be  separately  managed  (13  &  15).  The  Federal 
Supply  Class  identifies  which  commodity  group  the  item 
belongs  to  (15). 

A  subset  of  the  recoverable  spares  identified  by  the 
SMR  code  are  placed  on  the  Initial  Spare  Support  List 
(ISSL) . 
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ISSLs  include  those  provisioned  items  which  the 
equipment  specialist  and  the  using  command  agree  should 
be  stocked  and  on-hand  for  support  of  operations  and 
maintenance  at  the  operational  location  [emphasis 
added]  when  the  first  items  become  operational . 
(Reynolds,  1989:15) 

The  "operational  locations"  must  have  a  stock  quantity 
of  at  least  one  for  every  spare  identified  on  their  ISSL 
(24). 

Identifying  the  "range"  of  spares  required  is  only  a 
partial  fulfillment  of  the  initial  spares  provisioning 
objective.  The  "depth,"  or  number  of  spares  required  must 
also  be  computed.  AFLCR  57-27,  Initial  Requirements 
Determination ,  is  the  governing  regulation  for  determining 
the  quantity  of  new  spares  (those  not  already  stocklisted) . 
The  computations  associated  with  AFLC  Form  614,  "Recoverable 
Item  Initial  Requirements  Computation  Worksheet,"  are  the 
"standard"  methodology  for  determining  the  depth  of  required 
spares;  but  other  computerized  models  may  be  authorized  by 
AFLC/XRI  on  an  exception  basis  (20).  In  fact,  according  to 
one  source  (Dement,  1990:Sec  3),  XRII  (then  MMIE)  recommends 
the  use  of  the  Mod-Mult L-Echelon  Technique  for  Recoverable 
Item  Control  (Mod-METRIC)  model.  This  model,  discussed  in 
appendix  C,  was  used  on  both  the  F-15  (Huff,  1979)  and  F-16 
(27)  fighter  aircraft  programs. 

The  AFLC  Form  614  computations  make  use  of  provisioning 
factors  supplied  by  the  ALC  Equipment  Specialist  (16)  and 
information  relating  to  the  aircraft's  "operational  use" 
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supplied  by  the  Systems  Program  Manager  (18)  to  determine 
the  spares  requirements. 

The  provisioning  factors  are  based  upon  contractor 
input  and  the  knowledge  and  experience  of  the  Equipment 
Specialist.  The  following  factors  are  used  in  the  process 
of  determining  initial  spares  requirements: 

a.  Maintenance  factor, 

b.  Overhaul  replacement  percent, 

c.  Base  condemnation  percent, 

d.  Depot  condemnation  percent,  and 

e.  Not  reparable  this  station  (NRTS)  percent  (16). 

Maintenance  Factor .  The  maintenance  factor 
estimates  the  rate  of  demand  for  a  spare  part.  It  is 
defined  as  "the  estimated  average  maintenance  replacement 
rate  per  operating  program  increment  (OPI).  The  OPI  is 
either  considered  to  be  100  operating  hours  or  1000  rounds 
expended,  whichever  is  appropriate"  (16). 

Overhaul  Replacement  Percent .  The  overhaul 
replacement  percent  "represents  the  replacement  rate  for  a 
spare  or  repair  part  in  the  overhaul  of  the  next  higher 
assembly  (NHA)"  (17).  This  estimated  percentage  includes 
those  items  replaced  because  they  have  failed  and  those 
items  replaced  because  they  are  deemed  to  be  near  failure. 
This  is  analogous  to  an  auto  mechanic  wanting  to  change 
one's  oil  filter- -not  because  it  won’t  last  a  couple  hundred 
miles  more,  but  because  it  isn't  worth  risking  the 


154 


catastrophic  failure  of  one’s  engine  for  the  price  of  a  new 
oil  filter. 

Base  Condenmation  Percent.  This  factor  is,  as  the 
name  implies,  the  estimate  of  the  percentage  of  spares  which 
will  be  condemned  at  the  base  repair  level  (17). 

Depot  Condemnation  Percent .  This  factor  is  the 
estimate  of  the  percentage  of  spares  replaced  during  depot 
overhaul  which  are  condemned  (18). 

Not  Repairable  This  Station  (NRTS)  Percent .  This 
factor  "corresponds  to  the  portion  of  assets  which  are 
authorized  intermediate  level  repair  which  will  have  to  be 
returned  to  the  depot  for  repair  action"  (18).  The  causes 
for  this  vary  but  they  include  inadequate  equipment  and 
labor  skills  and/or  insufficient  capacity  at  the 
intermediate  levels  (18). 

As  stated  previously,  these  factors,  along  with 
operational  program  information  collected  by  the  System 
Program  Manager  on  checklists  such  as  the  AFLCR  57-27,  are 
then  used  in  computations  to  determine  spares  requirements. 
Figure  9  provides  examples  of  these  computations. 

The  AFLC  Form  614  computations  have,  historically,  been 
a  manual  process.  This  process  is  being  automated  on  a 
system  known  as  the  Initial  Requirements  Determination  (IRD) 
system.  The  IRD  is  currently  in  system  validation  test  and 
should  be  on  line  by  the  end  of  the  summer  of  1991  (Horner, 
1991) . 


155 


DLM  NJR  Annual  Demand  =  Average  Month  Program  x  12  x  QPEI  (for  engine  O/H 

or  prograiraned  depot  maintenance)  or  QPNHRA  (for 
MISTR  items)  x  Applications  %  x  NJR  Program  %  x 
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Figure  9.  Example  AFLC  Form  614  Requirements  Computations 
(AFLCR  57-27,  1986:  5-6) 


As  previously  stated,  the  AFLC  Form  614  process  is  used 
to  predict  requirements  for  spares  which  are  new  to  the 
inventory.  These  new  spares  are  "assigned  National  Stock 
Numbers  (NSNs)  for  purpose  of  identification  and  management" 
(11)  and,  if  AFLC  managed,  their  requirements  are  entered 
into  the  D041  system  for  determination  of  replenishment 
spares  requirements.  Some  spares  are  acquired  from  Non-AFLC 
agencies.  These  agencies  will  also  stocklist  their  new 
spares  for  managing  future  Air  Force  requirements  (27). 

Initial  spares  requirements  must  also  be  determined  for 
items  already  in  the  inventory.  For  those  "existing"  items 
which  are  AFLC  managed,  the  Item  Manager  uses  the  D041 
system  (used  for  replenishment  spares  calculations)  to 
determine  any  additional  quantities  required  to  support  the 
new  aircraft  (Horner,  1991;  Rosenthal,  1991).  The  Item 
Manager  may  use  the  AFLC  Form  614  worksheet  to  determine 
requirements  for  non-AFLC  managed  existing  items  and  must 
work  with  the  non-AFLC  agencies  to  determine  if  they  have 
adequate  stocks  for  these  parts  (27). 

Replenishment  Spares  Buy  Recfuirements  Determination 

Unlike  initial  spares,  where  the  budget  process  bares 
little  resemblance  to  the  execution  (provisioning)  process, 
replenishment  spares  budget  forecasts  are  based  upon  the 
same  process  (D041  calculations)  used  to  execute  the  budget. 
It  is  appropriate,  therefore,  to  reverse  the  order  of 
presentation  used  for  initial  spares  and  begin  this  section 


by  describing  the  D041  system  used  to  calculate 
replenishment  spares  buy  requirements.  Unless  stated 
otherwise,  references  in  this  section  will  come  from  a  June 
1987  AFLC  training  program  entitled  "Introduction  to  D041 
Requirements  System"  (course  #  LMMIM06). 

The  Recoverable  Consumption  Item  Requirements 

*• 

Computation  System  (D041)  actually  serves  several  functions. 

In  addition  to  determining  buy  requirements,  D041 
computations:  1)  assist  in  managing  depot  repair 
requirements,  2)  develop  Central  Secondary  Item 
Stratification  (CSIS)  lists  for  spares  budget  development, 

3)  determine  spares  contract  termination  requirements,  4) 
report  excess  requirements  for  disposal  action,  and  5) 
provide  for  control  of  distribution  of  the  spares 
requirements  (Ch  1,  5-6). 

Figure  10  provides  a  summary  level  view  of  the  basic 
requirements  computation  process  employed  by  D041.  The 
methodology  is  similar  to  that  used  for  initial  spares 
provisioning  (AFLCR  57-27)  in  that  factors  are  developed  to 
predict  future  requirements.  It  differs,  however,  in  that 
the  initial  spares  provisioning  factor  estimates  are 

typically  manually  developed  by  the  ALC  equipment  • 

specialists  based  upon  contractor  inputs  and  their  own 
expertise,  while  the  automated  D041  factors  can  be  updated 
based  on  historical  usage  recorded  during  the  initial  spares 
coverage  period  (an  average  of  two  years  worth  of  data). 

Figure  11  provides  examples  of  the  D041  requirements 
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*  USAGE  /  PAST  PROGRAM  =  FACTOR 

*  FACTOR  X  FUTURE  PROGRAM  =  PROJECTED  USAGE 


*  PROJECTED  USAGE 


+  STOCK  LEVELS 


+  WAR  READINESS  MATERIALS 


+  ADDITIVE  REQUIREMENTS 


=  GROSS  REQUIREMENT 


*  GROSS  REQUIREMENT 

-  ASSETS  {S-2rviceable  assets  plus  base  and 
depot  repai rabies) 


=  NET  REQUIREMENT 


Figure  10.  D0':1  Requirements  Determination 

Mei.hodology  (Ch  1,  2  &  Ch  7 ,  1) 


computation  factors.  These  factors,  developed  on  past 
usage,  are  then  applied  to  the  projected  future  flying 
programs  to  predict  future  spares  requirements.  Stock  level 
requirements,  war  readi.iess  material  requirements,  and 
additives  are  added  to  these  predicted  levels  to  arrive  at 
gross  requirements.  Serviceable  assets  on  hand  and  those 
projected  to  be  repaired  at  base  and  depot  facilities  are 
then  subtracted  to  arrive  at  the  net  buy  requirement  (Ch  1, 
1). 
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Figure  11.  Example  D041  Requirements  Factors 


The  D041  system  is  actually  at  the  center  of  numerous 
other  computerized  systems  that  feed  data  into  it.  Chapter 
10  of  AFLCR  57-4  describes  these  input  systems  in  detail. 
There  are  three  major  categories  of  input  data;  1) 
programmatic  information,  2)  historical  spares  usage  data, 
and  3)  inventory  assets  information.  The  "Worldwide  Stock 
Balance  and  Consumption  Report"  (SB&CR)  provided  by  the  D104 
system  provides  two  of  these  categories--usage  and  assets. 
Because  it  is  the  major  data  contributor,  the  data  cutoff 
dates  for  the  quarterly  D041  runs  are  aligned  with  the  SB&CR 
cutoff  dates--30  June,  30  September,  31  December,  and  31 
March  (Ch  2,  2) . 

Each  June  D041  cycle  contains  quarterly  spares 
requirements  forecasts  for  25  quarters  plus  a  three  year 
lump-sum  retention  estimate  (9  1/4  fiscal  years  total).  As 
the  fiscal  year  progresses,  each  passing  quarter  is  dropped 
from  the  computations  until  the  following  June  cycle  when 
another  fiscal  year's  (four  quarters)  forecast  is  added.  In 
this  way  the  number  of  quarters  included  in  the  forecast 
varies  from  25  to  22  (and  then  back  to  25)  (Ch  7,  7). 

For  budgeting  purposes,  separate  runs  known  as  the 
Central  Secondary  Item  Stratification  (CSIS)  runs  are 
computed.  Each  June  CSIS  contains  forecasts  for  the  June 
quarter  and  the  following  12  quarters.  The  number  of 
quarters  forecasted  varies  for  each  cycle  in  the  same  manner 
described  above  (Ch  7,  8). 
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These  automated  computations  arc;  reviewed  by  the 
inventory  management  and  equipment  sj)ecialists  at  the  ALCs 
for  error  correction  and  validation  l)efore  they  are 
formalized.  The  specialists'  interface  with  the  D041  system 
is  known  as  "file  maintenance"  (Ch  2,  1-2).  It  is  the 
inventory  management  specialist  who  is  ultimately 
responsible  for  a  computation’s  accuracy  and,  along  with  it, 
the  Air  Force's  support  posture  for  the  item  (Ch  1,  1). 

Replenishment  Spares  Budget  Devel opmtint 

As  stated  previously,  the  repl eiiishment  spares  budget 
forecasts  are  based  upon  the  D041  system  computations. 
Near-term  budgetary  requirements  such  as  the  President’s 
Budget  (PB)  and  Budget  Estimate  Submission  (BES)  inputs  are 
"scrubbed"  versions  of  the  D041  CSIS  reports.  According  to 
the  Repairable  Stock  Division  Program  Manager  at  HQ  AFLC, 
the  scrubbing  process  which  he  and  the  ALC  specialists 
perform  can  involve  numerous  assorted  modifications  to  the 
CSIS.  Besides  corrections  for  problems  such  as  data  entry 
errors  (a  few  stray  zeros  can  really  mess  up  a  forecast), 
changes  are  made  to  accommodate  Congressional  or 
Headquarters  Air  Force  direction  concerning  the  size  of  the 
budget.  In  one  instance,  the  Office  of  the  Secretary  of 
Defense  (OSD)  predicted  that  moving  the  replenishment  spares 
budget  to  a  stock  fund  concept  (DMRD  904)  would  save  10%  of 
the  budget  and  therefore  the  spares  judget  was  decremented 
by  10%.  In  general,  the  budget  is  "scrubbed"  so  that,  for 
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whatever  reason,  an  item's  budgetary  requirement  does  not 
fall  drastically  out  of  line  with  its  recent  execution 
experience.  For  example,  it  would  be  politically 
unacceptable  for  an  item  which  has  experienced  an  annual 
usage  requirement  of  one  million  dollars  for  five  straight 
years  to  suddenly  require  (per  D041)  a  ten  million  dollar 
annual  budget  (Rosenthal,  1991). 

Long  range  budget  requirements  for  the  Program 
Objective  Memorandum  (POM)  are  also  based  on  D041  CSIS 
reports.  The  Air  Logistics  Early  Requirements  Technique 
(ALERT)  (described  in  more  detail  in  Appendix  C)  has  been 
used  since  1984  to  develop  POM  inputs  (Rexroad,  Lucas,  and 
Collins,  1989:  1).  This  approach  combines  linear  regression 
and  expert  judgement  to  predict  out-year  requirements  based 
upon,  among  other  things,  the  aggregated  sum  (aircraft 
mission/design  level)  of  past  CSIS  reports. 

The  expert  judgement  comes  into  play  when  the 
Repairable  Stock  Division  Program  Manager  "scrubs"  the 
output  of  the  alert's  linear  regression-based  models.  This 
scrubbing  process  is  similar  to  that  described  earlier  but 
it  also  includes  adding  requirements  for  new  items  not 
included  in  the  D041  inventory.  In  those  situations  where 
long  range  forecasts  must  be  made  for  new  items  with  no 
historical  usage  database,  ALC  specialists  must  resort  to 
using  factor-based  computations  similar  to  those  prescribed 
in  AFLCR  57-27  (the  initial  provisioning  bible)  (Rosenthal, 
1991). 
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Relationship  of  Recoverable  Spares  to  Life  Cycle  Phase 

In  discussing  the  differences  between  the  initial 
spares  budgeting  and  provisioning  methodologies,  it  was 
briefly  mentioned  that  they  (the  differences)  resulted  from 
a  lack  of  available  data  during  the  earlier  budgeting  phase. 
This  section  will  provide  a  more  in  depth  analysis  of  the 
relationship  of  recoverable  spares  cost  estimating  to  an 
aircraft's  life  cycle  phase.  The  section  begins  with 
definitions  of  the  major  life  cycle  phases  and  milestones. 
Following  this  are  subsections  which  relate  these  phases  to 
their  effect  on  maintenance  plan  development,  cost 
estimating  methodologies,  and  the  different  reasons  for  cost 
estimating . 

Definitions  for  Life  Cycle  Phases /Milestones .  Figure 
12  illustrates  the  different  weapon  system  life  cycle 
phases.  Before  passing  from  one  phase  to  the  next,  new 
acquisition  programs  must  be  approved  at  major  milestone 
reviews  (shown  between  the  life  cycle  phases  in  Figure  12). 
Unless  specified  otherwise,  references  in  this  subsection 
will  come  from  Captain  Reynold's  1989  ISSL  study. 

The  DoD  components  (e.g..  Air  Force,  Army,  etc.) 
continuously  monitor  their  capabilities  in  relation  to  new 
mission  requirements.  Deficiencies  may  arise  for  numerous 
reasons  such  as  "obsolescence  of  existing  systems,  the 
development  of  new  technologies,  [and]  the  emergence  of  new 
threats"  (51  6  53).  Within  the  Air  Force,  the  various  major 
commands  identify  changes  in  their  operational  requirements 
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by  coordinating  (with  other  effected  major  commands)  and 
then  publishing  a  Statement  of  Operational  Need  (SON)  (53). 
The  Milestone  0  review  authorizes  the  exploration  of 
alternative  solutions  to  the  SON. 

Before  discussing  the  subsequent  phases  and  milestone 
reviews  a  brief  description  of  the  milestone  review  process 
may  be  helpful .  The  Air  Force  Systems  Acquisition  Review 
Council  (AFSARC)  reviews  the  programs  at  each  milestone  and 
makes  its  recommendations  to  the  Secretary  of  the  Air  Force 
(SAF).  The  DoD  assistant  secretaries  and  directors  then 
meet  at  the  Defense  Acquisition  Board  (DAB)  to  obtain 
Secretary  of  Defense  approval  for  continuing  the  program 
into  the  next  life  cycle  phase  (TASC,  1989:Ch  14,  7). 

During  the  concept  exploration  phase,  program 
management  solicits  industry  to  identify  solutions  to  their 
mission  need.  A  Request  For  Proposal  (RFP)  is  issued  that 
"includes  complete  information  concerning  the  mission  need, 
the  operational  and  threat  environments,  schedule  and  cost 
goals,  and  capability  objectives"  (55).  Each  contractor 
proposal  is  evaluated,  and  cost  estimates  are  developed  for 
each  alternative.  Although  the  program  is  still  in  its 
infancy,  the  decisions  made  as  the  result  of  these 
evaluations  will  typically  lock  in  about  70  percent  of  the 
program's  life  cycle  cost  (60).  For  exan.ple,  one 
alternative  may  include  developing  a  brand  new  weapon  system 
while  another  may  recommend  modifying  or  buying  more  of  an 
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existing  system.  Depending  on  which  route  (if  any)  is 
taken,  there  can  be  a  dramatic  impact  on  life  cycle  costs. 

Milestone  I  approval  allows  one  or  more  proposed 
programs  (if  funding  is  available)  to  proceed  into  the 
demonstration  and  validation  phase.  During  this  phase  paper 
studies  and/or  prototype  development  are  used  to  definitize 
the  system's  design  and  technology  (55).  Though  not  as 
dramatic  as  in  the  concept  exploration  phase,  the  tradeoff 
analyses  conducted  in  this  phase  still  have  a  great  impact 
on  life  cycle  cost.  At  the  end  of  this  stage,  approximately 
85  percent  of  the  cost  has  typically  been  fixed  (60). 

Milestone  II  approval  allows  the  system  to  enter  into 
full-scale  development  phase.  "During  this  phase,  the 
system,  including  all  essential  support  equipment  and 
documentation,  is  designed,  developed,  fabricated,  and 
tested"  (56).  Some  programs  are  also  granted  permission  to 
begin  low  rate  initial  production  (56).  By  this  time  the 
opportunities  to  impact  life  cycle  cost  have  been 
substantially  reduced  and  at  the  end  of  this  phase,  95 
percent  of  the  costs  have  already  been  locked  in  (60). 

Milestone  III  approval  "commits  the  Air  Force  to  buy 
the  system  for  operational  deployment  and  moves  the  system 
from  the  developmental  environment  into  full  production" 
(56).  It  is  during  this  phase  that 

mission  hardware,  support  equipment,  spares  and  repair 
parts,  personnel,  facilities,  etc.  necessary  to  operate 
and  maintain  the  system  are  produced,  acquired,  and 
assigned.  (56-57) 
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At  milestone  IV,  the  Logistics  Readiness  and  Support 
Review,  the  system  is  evaluated  to  see  if  it  is  functioning 
according  to  contractual  standards.  If  it  isn't,  this  is 
the  time  for  the  Air  Force  to  exercise  any  warranties  it  may 
have  included  in  the  contract.  At  this  point  the  weapon 
system  enters  the  longest  phase  of  its  life  cycle--the 
operations  and  support  phase.  There  is  typically  some 
overlap  of  the  production  and  operations  phases  because 
production  can  be  scheduled  over  many  years  (57). 

The  final  milestone  comes  at  that  point  in  time  when 
the  system  is  "no  longer  capable  of  meeting  the  operational 
need  for  which  it  was  acquired"  (57).  The  Major  Upgrade  or 
System  Replacement  Decision  (milestone  V)  decides  exactly 
what  its  title  says  it  does.  Some  systems,  such  as  the  B- 
52,  are  kept  in  the  operational  inventory  for  many  years 
through  major  modifications;  while  others  enter  the  final 
phase  of  the  acquisition  life  cycle- -the  disposal  phase. 

Maintenance  Plan  Development .  One  of  the  major 
difficulties  facing  a  life  cycle  cost  analyst  is  that  while 
tradeoff  decisions  made  during  the  earliest  phases  typically 
have  the  greatest  impact  on  life  cycle  CDsts,  the 
availability  of  data  on  which  to  base  the  decisions  is  very 
limited  during  this  period.  This  is  particularly  true  for 
Operations  &  Support  cost  analysts  who  must  attempt  to 
estimate  OSS  costs  based  on  immature,  ill-defined  system 
designs . 
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The  initial  objectives  for  logistics  support  are 
formulated  in  the  concept  exploration  phase  and  developed 
into  a  Integrated  Logistics  Support  Plan  (ISLP).  These 
objectives  are  refined  and  individual  ISLPs  are  developed 
for  each  alternative  approved  to  enter  into  the 
demonstration/validation  phase  of  the  contract  (TASC, 

1989;Ch  5,  8  &  11) , 

It  isn't  until  completion  of  the  full-scale  development 
phase,  however,  that  the  maintenance  plan  for  a  new  system 
is  fully  developed. 

Until  that  time,  the  predicted  parameters  and  desired 
characteristics  stated  in  the  Maintenance  Concept,  will 
not  have  been  validated.  It  is  virtually  impossible  to 
effectively  plan  for  Support  and  Test  Equipment, 

Spares,  Technical  Data,  Personnel  and  Training 
requirements  unless  the  projected  maintainability, 
reliability,  and  repair  level  criteria  for  a  given 
system  and  its  components  have  been  confirmed.  (TASC, 
1989:Ch  5,  55) 

It  is  obvious  from  the  above  quote  that  recoverable 
spares  are  but  one  of  many  considerations  which  occupy  the 
minds  of  logistics  planners.  It  is  also  clear  that  many  of 
the  factors  impacting  spares  cost  estimating 
(maintainability,  reliability  and  repair  level  criteria)  are 
not  confirmed  until  completion  of  the  full-scale  development 
phase.  It  is  important  to  understand  these  factors  in  more 
detail  because  they  are  important  cost  drivers  in 
recoverable  spares  cost  estimating. 

"Maintainability  refers  to  the  ease  with  which  a  given 
component/system  can  be  maintained"  (TASC,  1989:Ch  5,  18). 
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This  has  obvious  implications  for  maipower  costs  because  the 
"harder"  a  system  is  to  maintain,  thj  longer  it  typically 
takes  to  do  the  job.  This  is  also  related,  however,  to  the 
design  of  recoverable  spare  parts,  "^ewer  aircraft  such  as 
the  F-16  have  been  designed  with  modular  components  that  can 
be  quickly  removed  when  broken  and  replaced  with  serviceable 
spare  components  to  minimize  the  aircraft  maintenance 
downtime  (AFLCP  57-13,  1987:4). 

Reliability  is  another  major  0& 3  cost  driver.  It  is 
defined  as 

the  probability  that  the  system  will  satisfy  the  need 
for  which  it  was  intended  in  an  acceptable  manner,  for 
a  given  period  of  time,  when  deployed  and  used  under  a 
given  set  of  operating  conditions.  (TASC,  1989:Ch  5, 
16) 

It  is  easy  to  see  where  the  reliability  factors  must  be 
speculative  in  the  early  concept  exp!.oration  phase.  System 
specifications  which  define  "acceptaiile"  performance  may  not 
be  set  in  concrete  and  specific  operating  conditions  (e.g., 
temperature  cycles,  flying  hour  profiles,  vibration 
extremes,  etc.)  have  not  been  defini:ized  (TASC,  1989:Ch  5, 
16) . 

Reliability  is  quantified  throu?h  "terms  such  as:  mean 
time  between  failure  (MTBF) ,  mean  tine  between  maintenance 
(MTBM),  and  mean  time  between  repair  (MTBR)"  (TASC,  1989:Ch 
5,  17).  The  "maintenance  factor"  discussed  in  the  earlier 
section  on  initial  spares  provisioniig  factors  is  based  upon 
another  reliability  factor  known  as  mean  time  between  demand 
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(MTBD) .  While  contractor  data  provides  estimates  for  MTBF, 
historical  data  has  shown  these  estimates  to  be  "overly 
optimistic"  (Reynolds,  1989:81).  The  MTBD  is  a  derivative 
of  the  MTBF  which  has  been  modified  to  account  for  demand 
drivers  not  included  in  the  MTBF.  For  example,  the  MTBF 
does  not  include  component  failures  beyond  the  contractor's 
responsibility  such  as  operator  error  and  mishandling.  Not 
every  failure  leads  to  a  demand  and  this  also  is  accounted 
for  in  the  MTBD  (Reynolds,  1989:81-82). 

The  levels  of  maintenance  were  also  referred  to  in  the 
earlier  discussion  on  initial  spares  provisioning.  The  SMR 
code  identifies  what  repair  actions  are  permitted  at  what 
repair  level.  As  the  number  of  locations  with  authority  to 
perform  a  repair  action  increases,  greater  numbers  of  spares 
are  required  to  maintain  adequate  working  stocks  at  the 
different  repair  facilities.  "Level  of  Repair  policies  have 
undergone  continuous  development  and  change  since  the 
inception  of  the  use  of  aircraft  as  a  military  component" 
(TASC,  1989:Ch  5,  56).  While  there  were  four  echelons  of 
repair  in  World  War  II,  increased  maintenance  specialization 
requirements  reduced  this  number  to  the  three  level 
structure--organizational ,  intermediate  and  depot--used  by 
most  aircraft  today.  Today,  further  advances  in  the  need 
for  "highly  specialized  personnel,  test,  recalibration, 
repair  equipment,  and  specialized  facilities"  are  pushing 
new  systems  toward  a  "two  level  concept"  that  centralizes 
most  maintenance  at  the  depot  level,  "leaving  only  some 


circuit  boards),  and  pre¬ 


test,  plug-in  replacement  (i.e., 
flight  maintenance  at  the  organizational/f ield  level"  (TASC, 
1989:Ch  5,  57) . 

Cost  Estimating  Methodologies . 

The  availability  of  data  with  which  one  predicts  costs 
has  an  important  impact  upon  the  cost  estimating  methodology 
used.  The  following  major  classes  of  cost  estimating 
methods  will  be  briefly  discussed  in  regards  to  this  factor: 
1)  parametric,  2)  analogy,  and  3)  engineering  (or  "bottoms- 
up"). 

Parametric  Models .  Parametric  models  use 
mathematical  equations  known  as  cost  estimating 
relationships  (CERs)  to  identify  the  relationship  between  an 
item’s  cost  (the  dependent  variable)  and  one  or  more 
characteristics  of  that  item  (the  independent  variable(s)) 
(TASC,  1989:Ch  14,  25).  These  CERs  are  developed  by 
examining  relationships  exhibited  in  past  data,  and  their 
validity  rests  upon,  among  other  things,  the  assumption  that 
the  future  conditions  can  be  predicted  based  upon  this 
historical  data  set. 

CERs  can  be  useful  in  performing  trade-off  studies 
during  the  early  stages  of  an  acquisition  life  cycle. 
Although  final  system  designs  may  not  be  available, 
different  characteristic  values  may  be  input  into  the  model 
to  evaluate  their  effect  on  cost  and  design  goals  may  be 
substituted  for  firm  characteristics.  Great  care,  however. 


172 


must  be  given  to  ensure  that  the  CER's  provide  logical 
conclusions.  For  example  a  CER  might  show  that,  based  on 
the  sample  data,  engine  maintenance  costs  are  directly 
related  to  engine  size,  as  measured  in  thrust.  For  the  new 
engine  being  predicted  this  relationship  may  fail  because 

by  increasing  the  size  of  the  engine,  as  measured  in 
thrust,  the  engines  can  be  operated  at  derated  thrust 
levels  and  thereby  significantly  reduce  engine 
maintenance  costs.  (May,  1982 :Ch  3,  5). 

The  factor  based  approach  to  predicting  initial  spares 
requirements  discussed  earlier  is  an  example  of  a  very 
simple  CER.  In  many  cases,  such  as  this  one,  CERs  are 
relied  upon  when  insufficient  data  exists  to  conduct  more 
detailed  analysis. 

Analogy  Method.  This  methodology  depends  upon 
identifying  a  comparable  or  "analog.''ns'’  system  to  the  one 
being  evaluated.  Stated  simply. 

Analogy  estimating  begins  with  the  actual  cost  of  a 
similar  existing  system,  adjust  these  costs  for 
complexity,  technical,  or  physical  differences,  and 
then  derives  a  new  system  estimate.  (TASC,  1989:Ch  14, 
27) 

The  difficulty  in  this  method  is  in  finding  a  truly 
"similar"  system  and  then  knowing  how  to  adjust  costs  to 
account  for  the  differences  between  the  systems.  Typically, 
cost  analysts  must  seek  advice  from  technical  experts  to 
evaluate  these  differences  and  their  impact  on  cost. 

Finding  "experts"  who  are  familiar  with  both  the  analogous 
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system  and  the  new  system  can  be  difficult  and  the  "detailed 
program  and  technical  definition"  of  the  new  system  may  not 
be  available  during  the  initial  phases  of  the  acquisition 
life  cycle  (TASC,  1989:Ch  14,  27). 


Engineering  Estimation .  This  approach 
"encompasses  a  detailed  'build-up*  of  labor  hours,  material 
costs,  and  overhaul  at  very  finite  sub-indentures  of  the 
program/activity/item  for  which  cost  is  to  be  estimated" 
(TASC,  1989:Ch  14,  28).  As  such,  it  provides  the  most 
detailed  and  accurate  cost  estimates.  Due  to  its 
complexity,  however,  this  approach  also  requires  the 
greatest  amount  of  "calendar  time"  to  perform,  and  very 
detailed  system  data  must  be  available  (May,  1982:Ch  3,  8). 
Since  the  Maintenance  Plan  for  a  new  system  is  typically  not 
definitized  until  completion  of  the  full-scale  development 
phase,  the  data  required  for  this  approach  is  not  available 
until  that  time. 

It  is  frequently  the  case  that  a  combination  of 
methodologies  are  used  in  predicting  costs  (TASC,  1989:Ch 
14,  28-29).  For  example,  an  alternative  (recommended  by  two 
separate  studies)  to  the  factor  based  approach  currently 
used  in  estimating  initial  spares  requirements  is  using  the 
Logistics  Support  Cost  (LSC)  model  (Dement,  1990;  Rexroad, 
Tillia,  and  Tritle,  1990).  The  LSC  model  is  an  engineering 
or  "bottoms-up"  model.  Using  this  approach  during  early 
life  cycle  phases  requires  that  data  from  analogous  systems 
are  modified  and  input  into  the  model  (Dement,  1990 :Ch  2). 
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By  combining  approaches  in  this  way  one  can  work  around  some 
of  the  limitations  unique  to  the  individual  methodologies. 

Purposes  for  Cost  i:stimates .  The  cost  estimating 
methodologies  are  not  the  only  thing  which  may  vary 
depending  upon  the  ecqulsition  life  cycle  phase,  the  parpose 
behind  the  cost  est  .mate  also  varies. 

The  Program  Cost  Ei-timate  (PCE),  also  known  as  the 
Program  Office  Estimate  (POE),  is  the  "program  manager's 
official  estimate  o:  th ;  financial  resources  required  to 
competently  conduct  the  program  contained  in  its  Program 
tianagement  Directive  (P'lD)"  (TRSC,  1989:Ch  14,  8).  PCEs  are 
maintained  throughout  tie  system’s  life  cycle  and  are  used 
in  all  the  program's 

formal  reports--baseiines ,  Selected  Requisition  Report 
(SARs),  Program  Assessment  Reviews  (PARS),  Command 
Assessment  Reviews  (CARs),  Secretarial  Program  Reviews 
(SPRs),  and  financial  reviews.  (TASC,  1989:Ch  14,  8). 

An  Independent  Cost  Analysis  (ICA)  is  a  separate 
estimate  performed  by  an  "honest  broker"  with  no  ties  to  the 
program  office.  It  is  used  as  a  check  for  the 
reasonableness  of  the  PCE.  "ICAs,  by  law,  (DoD 
Authorization  Act  of  1984)  must  be  performed  on  major 
defense  acquisitions  in  support  of  Milestone  Decisions" 
(TASC,  1989:Ch  14,  8).  ICAs  are  briefed  to  the  Air  Force 
Cost  Analysis  Improvement  Group  (CAIG)  which,  in  turn,  pass 
their  recommendations  along  for  AFSARC  milestone  decisions. 
They  are  also  briefed  to  the  OSD  CAIG  which  makes  inputs  to 
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DAB  milestone  reviews.  ICAs  are  also  perrormed  on  selected 
programs  during  the  POM  process.  These  estimates  are  known 
as  Defense  Resource  Board  (DRB)  ICAs  (TAS:,  1989:Ch  14,  8). 

Additional  estimates  include:  1)  The  Should  Cost 
Estimate  (SCE)  which  is  developed  for  ind.  vidual  contracts 
as  a  basis  for  the  Government's  negotiation  objective;  2)  A 
Most  Probable  Cost  is  used  during  source  selection  to  check 
the  reasonableness  of  bidder's  cost  proptsals;  and  3)  the 
Single  Best  Estimate  results  when  the  IC  and  PCE  processes 
are  combined  to  arrive  at  one  estimate.  Both  program  office 
and  staff  cost  analysts  combine  to  form  joint  estimating 
team  (TASC,  1989:Ch  14,  9). 

Data  Sources  for  Recoverable  Spares  Cost  Estimating 

Having  discussed  the  different  types  of  cost  estimating 
methodologies  and  the  numerous  purposes  for  which  they  are 
developed,  it  is  appropriate  to  address  the  data  sources 
available  for  developing  recoverable  spa. es  estimates.  The 
following  remarks  are  based  upon  the  res< arch  conducted  ir 
support  of  this  thesis.  The  sources  wil  be  discussed  in 
the  following  order:  1)  D041,  the  "Recovirable  Consumptioi 
Item  Requirements  System;"  2)  H036C,  the  "Weapon  System  Cost 
Retrieval  System"  (WSCRS);  3)  Visibility  and  Management  of 
Operating  and  Support  Costs  (VAMOSC);  an  4)  Aeronautical 
Systems  Division  (ASD)  Cost  Library. 

D041  contains  a  tremendous  amount  of  recoverable  spares 
data  (e.g. ,  usage  and  inventory  data,  reliability  factors. 
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item  demand  rates,  condemnations,  unit  costs,  etc.)*  There 
were  two  major  obstacles  to  using  this  data  source  for  this 
thesis,  however.  The  primary  obstacle  was  the  fact  that 
D041  controls  individual  spares  at  the  National  Stock  Number 
(NSN)  level.  Since  each  aircraft  contains  thousands  of 
individual  parts,  it  was  impractical  to  attempt  manual 
aggregation  of  this  data.  The  D041  system’s  Central 
Secondary  Item  Stratification  (CSIS)  reports,  used  for 
budget  development,  do  provide  spares  requirements 
aggregated  to  the  Mission  Design  level.  "Scrubbed"  versions 
of  these  reports  were  obtained  from  the  Repairable  Stock 
Division  Program  Manager  (HQ  AFLC/FMBSR) . 

Other  analysts  have  attempted  to  use  the  D041  data  and 
encountered  the  second  major  obstacle — D041  was  not  designed 
to  be  a  cost  analysis  tool  and  the  mainframe  based  program 
is  not  "user  friendly."  Very  little  computer  time  is 
allotted  to  system  inquiries  and  "what  if"  exercises.  For 
those  brave  enough  to  attempt  analysis  with  D041,  data  is 
provided  on  large  nagnetic  tapes.  In  one  instance,  analysts 
devoted  a  large  amount  of  effort  to  learning  the  D041 
software  program  and  interpreting  the  data  they  received  on 
the  magnetic  tapes  only  to  find  that  there  were  gaps  in  the 
data  rendering  it  useless  (Dement,  1991). 

The  Weapon  System  Cost  Retrieval  System  (WSCRS)  was  a 
better  source  of  data  for  this  thesis.  The  WSCRS  system 

collects  and  assembles  the  historical  depot  maintenance 

cost  expenditures,  and  the  base  level  and  depot  level 
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condemnation  cost  expenditures,  for  the  major  USAF 
aircraft  and  missile  weapon  systems.  .  .  .  WSCRS  also 
collects  and  assembles  historic \1  weapon  system  program 
data  by  weapon  system  standard  IDS.  The  weapon 
system's  actual  and  programmed  itilization  data  are 
collected  by  fiscal  year.  (AFL ’M  173-264,  1990:7) 

Unlike  D041,  the  WSCRS  system  w is  originally  designed 
and  is  maintained  by  HQ  AFLC/FMC  to  support  cost  analysis 
projects  (AFLCM  173-264,  1990:5).  R  iquest  for  WSCRS  data 
(which  begins  with  FY  1975)  must  be  addressed  to  HQ  AFLC/FMC 
and  the  data  is  provided  in  hard  copy.  Several  different 
aggregate  levels  of  data  can  be  requested,  including  mission 
design  series  (MDS) ,  fleet  (MD) ,  and  mission  (i.e.,  fighter, 
bomber,  cargo,  etc.).  Data  can  be  further  divided  to  weapon 
system  work  breakdown  structure  (Stelnlegy,  1991). 

The  VAMOSC  system,  as  its  name  implies,  was  originally 
designed  in  the  mid-1970's  to  provide  increased  management 
insight  into  weapon  system  costs  (May,  1982 :Ch  5,  13). 
Through  the  years  this  system  has  been  modified  and  updated 
in  order  to  fulfill  its  original  promise.  Several  factors 
have  impaired  this  process  and  the  system  is  still  in  need 
of  maturation. 

Responsibility  for  VAMOSC  has  sviitched  hands  on 
different  occasions.  The  Air  Force  Cost  Center  is  the 
present  OPR  but  they  are  looking  for  an  alternative  host 
site.  Manpower  and  facilities  limitations  have  impaired 
VAMOSC  development  at  the  Cost  Center.  When  researched  for 
this  thesis,  only  three  quarters  of  data  were  input  into  the 
VAMOSC  automated  system.  Past  history  at  the  NSN  level  was 
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available  on  microfiche;  but  given  the  thesis  time 
constraints,  this  formidable  aggregation  task  was  even  less 
appealing  than  the  D041  system  (Masserro,  1991). 

The  ASD  Cost  Library  was  also  briefly  evaluated.  It 
appeared  to  be  an  excellent  source  for  identifying 
historical  cost  methodologies  used  by  various  programs  but 
it  contained  very  little  cost  data  with  which  to  conduct 
original  research. 
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Appendix  Models  Which  Predict  Recoverable  Spares  Costs 


Introduction 

The  purpose  of  this  appendix  is  to  evaluate  selected 
models  which  can  be  used  to  predict  recoverable  spares 
costs.  The  intent  is  not  to  provide  in-depth  operating 
instructions  but  rather  to  familiarize  the  reader  with  the 
models  and  their  applicability.  While  not  exhaustive,  the 
appendix  includes  both  engineering  or  "bottoms-up"  models 
and  parametric  models  selected  for:  1)  their  general  (as 
opposed  to  SPO  specific)  applicability,  and  2)  their 
prominent  use  in  the  AFLC  community  [one  parametric  model, 
developed  by  Rand  Corporation,  was  included  even  though  it 
is  not  in  "prominent"  use.  It  was  evaluated  simply  because 
there  were  few  parametric  models  in  the  literature  review 
and  this  model  was  evaluated  by  the  AFLC  "Initial  Spares 
Working  Group"  as  a  potential  candidate  for  predicting 
initial  spares  requirements].  The  text  begins  with  the 
analyses  for  the  following  parametric  models:  1)  Rand  study 
2)  Modular  Life  Cycle  Cost  Model  (MLCCM) ;  and  3)  Air 
Logistics  Early  Requirements  Technique  (ALERT).  Next,  the 
following  engineering  models  are  eVfiluated:  1)  D041,  the 
"Weapon  System  Cost  Retrieval  System;"  2)  AFLC  Form  614,  the 
initial  spares  requirements  computal.ion  form;  3)  Logistics 
Support  Cost  (LSC)  Model;  4)  Mod-Multi-Echelon  Technique  for 
Recoverable  Item  Control  (Mod-METRIC) ;  and  5)  Dyna-METRIC. 
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A  standard  format  is  used  during  each  analysis  to  simplify 
comparisons  between  the  models. 

Rand  Study 

All  references  in  this  subsection,  unless  identified 
otherwise,  come  from  the  developer’s  (see  below)  1980 
report ,  Estimating  USAF  Aircraft  Recoverable  Spares 
Investment . 

Developer (s) ;  K.J.  Hoffmayer,  F.W,  Finnegan,  Jr.,  and 
W.H.  Rogers  of  the  Rand  Corporation,  August  1980. 

Model  Purpose :  The  model  is  an  update  to  a  previous 
1976  Rand  model  for  estimating  USAF  aircraft  recoverable 
spares  investment.  It  includes  models  for  estimating  total 
replenishment  spares  requirements  at  the  major  subsystem 
level  (airframe,  avionics,  and  propulsion)  and  for 
estimating  condemnation  spares  requirements  at  the  same 
level  (v) .  An  attempt  was  made  to  develop  an  initial  spares 
model  but  this  was  unsuccessful  due  to  limited  data 
availability  (23).  The  models  provide  annual  estimates  for 
peacetime  operating  stock.  War  readiness  material,  spare 
engines,  and  engine  spare  parts  are  excluded.  The  models 
are  intended  for  use  prior  to  the  preproduction  or 
deployment  decision  stages  of  the  acquisition  life  cycle 
(iii) . 

Model  Algorithm;  The  study  states  that  a  logarithmic 
form  was  chosen  to  develop  the  CERs  due  to  its  (logarithmic 
CERs  in  general)  superior  handling  of  heteroscedasticity  and 
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its  more  "real  world"  multiplicative  nature  (11,  15).  They 
fail  to  clearly  state  whether  the  transformations  are 
natural  (In)  or  common  (log)  logarithmic  in  nature. 

Table  17  shows  the  cost  drivers  used  on  each  CER. 


Table  17 

Subsystem  CER  Independent  Variables  (12-14) 


1  Airframe  CER 

Avionics  CER 

Propulsion  CER 

1  Total  active 

1  aircraft  in  MDS 
inventory 

Total  active 
aircraft  in  MDS 
inventory 

Total  number  of 
installed  engines 
in  the  MDS  force 

j  Airframe  flyaway 
i  cost 

Avionics  flyaway 
cost 

Propulsion 
flyaway  cost 

Peak  flying  hours 
per  MDS  per  year 

Dummy  variable  for 
bomber 

Dummy  variable  for 
reconnaissance 

Dummy  variable  for 
f ighter/attack 

Dummy  variable  for 
cargo 

1 

Dummy  variable  for 
tanker 

The  following  logarithmic  form  is  common  to  each  of  the 
subsystem  CERs  (11): 


log  =  log  a  log  X^j-^  +  (12) 

Where 


=  investment  in  PCS  spares  inventory  of  aircraft 
subsystem  i  at  time  t. 

=  the  jth  characteristic  observed  on  aircraft 
subsystem  i  at  time  t. 

a  and  Pj  =  regression  coefficients 
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Bjj  =  the  error  for  aircraft  subsystem  i  fit  time  t. 

The  errors  are  assumed  to  be  independent 
across  subsystems  but  correlated  over  time  with 
subsystem. 


Data  Inputs  and  Sources :  Cost  data  from  1975  to  1978 
was  provided  for  the  following  aircraft  (7): 


A-7D 

C-5A 

RF-4C 

F-lllD 

B-52D 

KC-135A 

F-4D 

F-lllF 

B-52G 

C-141A 

F-4E 

T-37B 

B-52H 

F-4C 

F-lllA 

T-38A 

Specific  data  elements  and  sources  are  as  follows  (4-5): 

D041  "Recoverable  Consumption  Item  Requirements  System:" 

National  Stock  Number  (NSN) 

Unit  Price  (in  then-year  dollars) 

Program  Begin  Date  (earliest  record  of  use) 

Program  Selection  Code  (Material  Program  managing  part) 
Organization  Field  Maintenance  (OFM)  Total  Demand 
Rate  (total  item  demand  expressed  in  terms 
appropriate  for  its  material  program) 

Base  bevel  Condemnations  (NSN  level  condemnations  at 
base  level) 

Depot  Level  Condemnations  (NSN  level  condemnations  at 
depot  level) 

Total  Overhaul  Condemnations  (NSN  level  condemnations 
resulting  from  planned  overhauls) 

Total  Peacetime  Operating  Stock  Assets 
Application  (the  mission  design  series  (MDS)  or  other 
stock  number  using  the  item) 

Quantity  Per  Application 

J041  "Procurement  History  File:" 

National  Stock  Number  (NSN) 

Contract  Date 
Amount  of  Contract  ($) 

Quantity  Procured 

"Aerospace  Vehicle  Inventory  Status  and  Utilization  and 
Reporting  System"  (AVISURS): 

Aircraft  MDS 

Calendar  Year  and  Month 

Flying  Hours 

Sorties 

Landings 

Average  Number  of  Possessed  Aircraft 
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other  data  used  in  the  model  development  was  obtained  from 
the  following  sources: 

*  TO  0025-30,  Technical  Manual,  "Unit  Cost  of 

Aircraft,  Guided  Missiles,  and  Engines." 

*  USAF  Statistical  Digests 

*  PA,  "USAF  Program,  Aerospace  Vehicles  and  Flying 

Hours." 

Assessment ;  The  accuracy  of  the  models  provided  are 
difficult  to  assess  due  to  the  minimal  coverage  of  model 
diagnostics.  The  provided  for  the  propulsion  model  has 
poor  statistics:  of  .5841  and  SEE  of  .67318.  Because 

Rand  does  not  specify  which  base  was  used  (natural  or 
common)  the  reader  is  left  to  guess  at  the  significance  of 
the  standard  error  of  the  estimate  (the  SEE  is  a  measure  of 
prediction  accuracy  for  a  transformation  using  the  natural 
base) (Murphy ,  1990-1991). 

It  is  unclear  if  the  logarithmic  form  is  really 
appropriate  because  they  never  actually  state  that  they 
observed  heteroscedasticity  in  the  data;  or  why  they  feel  a 
multiplicative  equation  is  more  "real  world".  They  fail  to 
document  any  diagnostics  performed  or  other  models  attempted 
and  discarded. 

The  manner  in  which  subsystem  flyaway  costs  were 
included  in  each  of  the  subsystem  CERs  may  be  questioned. 

One  tenet  of  the  integrated  logistics  support  ph5.1osophy  is 
that  as  an  item's  reliability  improves,  the  reduction  in  its 
operations  and  support  costs  over  its  life  cycle  more  than 
compensates  for  its  increased  acquisition  cost  (which  has 
resulted  from  its  improved  reliability)  (TASC,  1989:Ch  5, 
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25).  The  multiplicative  nature  of  the  subsystem  CERs  does 
not  account  for  this  belief  logically.  According  to  these 
CERs,  increased  CERs  will  always  result  in  greater  spares 
costs . 

Although  the  source  text  does  not  explain  its  rationale 
for  using  component  flyaway  costs,  a  case  can  be  made  that 
life  cycle  O&S  costs  reduce  with  improved  reliability  for 
reasons  other  than  reduced  spares  requirements.  After  all, 
replenishment  spares  costs  are  only  a  subset  of  the  total 
OSS  costs.  More  reliable  parts  should  fail  less  often  and 
therefore  cost  less  for  maintenance  and  repair.  If  RAND 
believes  that  these  types  of  savings  exceed  the  increased 
spares  costs  associated  with  more  expensive  components,  then 
their  logic  is  not  contradictory  to  reliability  theory. 

This  methodology  was  evaluated  by  the  Initial  Spares 
Working  Group  comprised  of  twenty  two  members  representing 
HQ  AFLC,  ASD  and  HQ  AFS3.  They  concluded  that  the  model’s 
database  should  be  updated  and  a  clear  distinction  made 
between  initial  and  replenishment  spares  before  they  could 
use  the  model  (Rexroad,  Tillia,  and  Tritle,  1990:11). 

Modular  Life  Cycle  Cost  Model  ( MLCCM ) 

All  references  in  this  subsection,  unless  specified 
otherwise,  come  from  Grumman  Aerospace  Corporation's  1986 
report  entitled  Modular  Life  Cycle  Cost  Model  for  Advanced 
Aircraft  Systems ,  Cost  Methodology  Development  and 
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Application.  The  authors  were  R.  Isaacs,  N.  Montanaro,  and 
F.  Olivo. 

Developer  (s') ;  Grunman  Corporation,  Program  Team 
directed  by  Mr.  R.  Isaacs,  September  1986. 

Model  Purpose:  The  MLCCM  is  a  parametric-based  series 
of  models  for 

predicting  advanced  technology  aircraft  costs,  to  the 
major  subsystem  levels,  for  the  Research,  Development, 
Test,  and  Evaluation,  Production,  Initial  Support,  and 
Operations  and  Support  phases  of  the  system  life  cycle 
during  conceptual  and  preliminary  design.  (iii) 

Initial  and  replenishment  spares  are  but  subsets  of  the 
overall  costs  within  the  production  and  OSS  periods, 
respectively.  Their  cost  is  broken  out  for  14  subsystems 
(structure,  crew  system,  landing  gear,  flight  control,  cargo 
handling,  engines,  engine  installation,  envirtnmental 
control  systems,  electrical,  hydraulic/pneumatic,  fuel 
system,  avionics,  armament,  auxiliary  power  unit)  (11)  for 
two  classes  of  aircraft  (fighter/at :ack/bomber  and 
cargo/transport/  tanker) (x). 

Model  Algorithm:  Because  the  MLCCM  was  developed  as  a 
tool  for  conducting  trade  studies  during  the  design  stage, 
the  CERs  were  developed  using  a  Work  Breakdown  Structure 
(WBS)  format.  In  this  way  the  design  engineers  would  be 
able  to  relate  costs  to  the  WBS  elements  for  which  they  were 
responsible.  Step-wise  regression  was  used  to  develop  log- 
linear  regression  equations.  Again,  it  is  unclear  if  they 
used  natural  or  common  base  transformations.  They  limited. 
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for  most  cases,  the  number  of  parameters  in  any  CER  to  one 
third  the  number  of  data  points  (47-48).  While  it  makes 
sense  to  preserve  degrees  of  freedom  by  limiting  the  number 
of  variables  used  (compared  to  the  number  of  data  points), 
it  isn't  clear  why  the  developers  chose  one  third  as  the 
criteria  ratio.  The  developers  provide  no  explanation  for 
this  criteria. 

Data  Inputs  and  Sources :  Cost  data  was  derived  from 
several  sources  including:  "Visibility  and  Management  of 
Operating  Support  Costs"  (VAMOSC)  system,  AFR  173-13 
Factors,  and  the  1975/1976  Operating  and  Support  Cost 
Estimating  Report .  Independent  variable  technical  data 
sources  include:  Standard  Aircraft  Characteristics  (SAC) 
charts,  group  weight  statements,  technical  orders,  and  the 
manufacturer. 


The  SAC  charts  were  used  to  obtain  data  on  engine 
design  and  performance,  fuel  and  tankage,  armament, 
loading  and  aircraft  performance,  development  dates, 
etc.  Weights,  areas,  volumes,  dimensions,  and  general 
aircraft  design  characteristics  were  obtained  from  the 
group  weight  statements  and  the  manufacturers.  Flight 
manuals  were  used  for  data  on  electrical,  fluid  power, 
and  flight  actuator  systems  and  for  general  aircraft 
design  characteristics  as  well  [note  that  not  all  of 
these  variables  are  used  in  the  replenishment  spares 
CERs].  (13) 


Assessment :  The  fact  that  the  models  were  developed 
with  obligation  data  brings  with  it  all  the  uncertainty 
previously  discussed  in  the  data  problem  section.  The 
report  admits  the  need  for  better  cost  data  inputs  (219). 
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The  model  statistics  provided  were  not  complete.  R 
values  were  provided  as  opposed  to  the  R  statistic  commonly 
seen.  The  R^  values  for  5  of  the  14  subsystems  in  the 
fighter/attack/bomber  class  of  aircraft  were  poor  (less  than 
.7)  (165-169).  No  discussion  of  model  diagnostics  was 
provided. 

Rir  Logistics  Early  Regixirements  Technique  (P.LERT) 

Unless  stated  otherwise,  references  in  this  subsection 
will  come  from  a  1989  report  entitled  Air  Logistics  Early 
Requirements  Technigue  (ALERT)  FY90-94  Program  Objective 
Memorandum  ( POM)  Forecast .  The  authors  were  Adrienne 
Rexroad,  Robert  Lucas,  and  Larry  Collins. 

Developer(s) t  AFLC/MMM,  1984. 

Model  Purpose;  ALERT  has  been  used  since  1984  by  HQ 
AFLC  as  the  starting  point  for  developing  BP15  aircraft 
peacetime  spares  Program  Objective  Memorandum  (POM)  inputs. 
It  is  the  "starting  point"  because  the  output  from  ALERT  is 
scrubbed  by  the  BP15  Program  Manager  prior  to  its  submittal 
(1). 

Model  Algorithm;  The  CERs  developed  used  straight 
linear  regression  to  predict  the  first  year  of  requirements. 
This  first  year's  estimate  is  then  used  as  historical  input 
for  the  next  four  years'  predictions — a  regression  technique 
which  referred  to  as  "bootstrapping."  (2). 

Data  Inputs  and  Sources ;  The  dependent  variable  data 
used  were  the  requirements  submitted  in  the  last  Budget 
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Estimate  Submission  (BES) .  There  were  a  total  of  four 
independent  variables  used  (not  all  at  once)  in  the  CERs:  1) 
Mission  Design  Series  buy  requirements  from  the  D041 
"Recoverable  Consumption  Item  Requirements  System,"  2) 
Average  Fleet  Value,  as  calculated  by  USAF/AC,  3)  the 
reciprocal  of  the  estimated  Present  Fleet  Age,  also  provided 
by  USAF/AC,  and  4)  Chronological  Year. 

Assessment :  Only  six  of  the  sixteen  weapon  system 

class  CERs  had  adjusted  R  values  exceeding  .7  and  only  one 
class  exceeded  .75.  The  BP  Manager  scrub  that  followed 
subsequent  to  the  ALERT  run  changed  the  input  values 
further.  The  BP  Manager  is  critical  of  the  D041  input  data 
since  it  uses  the  June  data  run  (a  quarter  not  updated  by 
three  of  the  five  ALCs).  He  also  questioned  the  logic  of 
using  the  fleet  value  as  a  cost  driver.  USAF/AC  based  their 
estimate  of  fleet  value  on  projected  future  flying  hour 
programs  which  decrease  over  time.  The  fleet  values, 
therefore,  decrease  over  time.  The  spares  requirement, 
however,  logically  gets  larger  as  the  fleet  gets  older  (7). 

Unless  stated  otherwise,  references  in  this  subsection 
will  come  from  a  June  1987  AFLC  training  program 
entitled  "Introduction  to  D041  Requirements  System"  (course 
#  LMMIM06). 

Developer(s) :  Unidentified.  Managed  by  HQ  AFLC/MM 
(Material  Management  Division) . 


Model  Purpose;  The  Recoverable  Consumption  Item 


Requirements  Computation  System  (D041)  actually  serves 
several  functions.  In  addition  to  determining  buy 
requirements,  D041  computations:  1)  assist  in  managing  depot 
repair  requirements,  2)  develop  Central  Secondary  Item 
Stratification  (CSIS)  lists  for  spares  budget  development, 

3)  determine  spares  contract  termination  requirements,  4) 
report  excess  requirements  for  disposal  action,  and  5) 
provide  for  control  of  distribution  of  the  spares 
requirements  (Ch  1,  5-6). 

D041  can  hardly  be  classified  as  simply  a  cost  model. 
Because  it  is  the  system  used  to  execute  the  replenishment 
spares  budget,  however,  other  cost  models,  such  as  the 
Logistics  Support  Model,  are  compared  to  D041  to  validate 
their  results  (Alexander,  Brcokey,  Erhart,  Fulton,  Hofmann, 
and  Shutak,  1990:1-2;  Dement,  1990).  For  this  reason  it  is 
included  in  the  appendix 

Model  Algorithm:  Figure  13  provides  a  svimmary  level 
view  of  the  basic  requirements  computation  process  employed 
by  D041.  The  methodology  is  similar  to  that  used  for 
initial  spares  provisioning  (AFLCR  57-27)  in  that  factors 
are  developed  to  predict  future  requirements.  It  differs, 
however,  in  that  the  initial  spares  provisioning  factor 
estimates  are  typically  manually  developed  by  the  ALC 
equipment  specialists  based  upon  contractor  inputs  and  their 
own  expertise,  while  the  automated  D041  factors  can  be 
updated  based  on  historical  usage  recorded  during  the 
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*  USAGE  /  PAST  PROGRAM  =  FACTOR 

*  FACTOR  X  FUTURE  PROGRAM  =  PROJECTED  USAGE 


*  PROJECTED  USAGE 


+  STOCK  LEVELS 


+  WAR  READINESS  MATERIALS 


+  ADDITIVE  REQUIREMENTS 


=  GROSS  REQUIREMENT 


*  GROSS  REQUIREMENT 

-  ASSETS  (Serviceable  assets  plus  base  and 
depot  repairables) 


=  NET  REQUIREMENT 


Figure  13.  D041  Requirements  Determination 

Methodology  (Chi,  2&Ch7,  1) 


initial  spares  coverage  period  (an  average  of  two  years 
worth  of  data).  Figure  14  provides  examples  of  the  D041 
requirements  computation  factors.  These  factors,  developed 
on  past  usage,  are  then  applied  to  the  projected  future 
flying  programs  to  predict  future  spares  requirements. 

Stock  level  requirements,  war  readiness  material 
requirements,  and  additives  are  added  to  these  predicted 
levels  to  arrive  at  gross  requirements.  Serviceable  assets 
on  hand  and  those  projected  to  be  repaired  at  base  and  depot 
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facilities  are  then  subtracted  to  arrive  at  the  net  buy 
requirement  (Ch  1,  1). 

Data  Inputs  and  Sources;  The  D041  system  is  actually 
at  the  center  of  numerous  other  computerized  systems  that 
feed  data  into  it.  Chapter  10  of  AFLCR  57-4  describes  these 
input  systems  in  detail.  There  are  three  major  categories 
of  input  data:  1)  programmatic  information,  2)  historical 
spares  usage  data,  and  3)  inventory  assets  information.  The 
"Worldwide  Stock  Balance  and  Consumption  Report"  (SB&CR) 
provided  by  the  D104  system  provides  two  of  these 
categories--usage  and  assets.  Because  it  is  the  major  data 
contributor,  the  data  cutoff  dates  for  the  quarterly  D041 
runs  are  aligned  with  the  SB&CR  cutoff  dates--30  June,  30 
September,  31  December,  and  31  March  (Ch  2,  2). 

Assessment :  It  is  difficult  to  evaluate  D041  without 

examining  the  many  conti ibuting  systems  which  feed  into  it. 
For  example,  the  Modified  Dyna-METRIC  model  provides  factors 
used  by  D041  for  determining  War  Readiness  Spares  Kits 
(WRSKs)  (Oster,  Sakulich,  and  Stone,  1989:3).  Additionally, 
D041  uses  inputs  from  the  Aircraft  Availability  Model  (AAM) 
to  determine  its  base  safety  level  stock  (Alexander, 

Brookey,  Erhart,  Fulton,  Hofmann,  and  Shutak,  1989:IX-16). 

As  a  result,  D041  snares  in  the  same  benefits  and  drawbacks 
associated  with  its  input  systems.  In  the  preceding 
examples,  both  models  allow  D041  to  optimize  the  repair 
capability  gained  for  these  spares  categories  within  a  given 
budget  constraint.  Marginal  analysis  techniques  are 


utilized  to  examine  the  relationship  between  final 
assemblies  (e.g.,  a  "black  box"),  known  as  the  line 
replaceable  units  (LRUs),  and  their  various,  less  expensive 
components  (e.g.,  a  circuit  board),  known  as  shop 
replaceable  units  (SRUs)  (Alexander,  Brookey ,  Erhart , 

Fulton,  Hofmann,  and  Shutak,  1989:IX-16;  Oster,  Sakulich, 
and  Stone,  1989:2).  If  the  components  within  an  assembly 
that  are  most  likely  to  fail  can  be  identified,  it  is  more 
economical  to  stock  a  number  of  the  SRUs  rather  than 
purchasing  numerous,  more  expensive  LRUs  to  arrive  at  the 
same  amount  of  repair  capability. 

As  previously  stated,  the  D041  system  is  the  standard 
by  which  other  models  are  evaluated  for  their  validity. 

That  is  not  to  say  that  it  is  a  perfect  system  by  any  means. 
The  D041  product  used  for  budget  development,  known  as  the 
Central  Secondary  Item  Stratification  (CSIS)  report,  must  be 
thoroughly  scrubbed  by  the  Repairable  Stock  Division  Program 
Manager  and  the  ALC  inventory  management  and  equipment 
specialists.  According  to  the  Repairable  Stock  Division 
Program  Manager,  it  is  not  uncommon  for  the  CSIS  reports  to 
contain  numerous  errors  (e.g.,  data  entry  problems)  that, 
before  being  "scrubbed,"  predict  requirements  which  are 
substantially  off  the  mark  (Rosenthal,  1991). 

The  D041  system  was  also  not  designed  as  a  cost 
estimating  tool .  It  manages  recoverable  spares  down  to  the 
Federal  Stock  Number  level  and  it  can  aggregate  requirements 
to  the  aircraft  mission  design  level.  It  has  no  built  in 
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capability  to  aggregate  its  data  to  other  levels  (such  as 
work  unit  code,  for  example).  For  this  reason  it  is 
impractical  to  think  of  the  D041  as  a  cost  estimating  tool . 
It  is  too  inflexible  to  perform  what-if  type  exercises  and 
very  little  computer  time  is  made  available  for  system 
inquiries  (Artley,  1991). 

AFLC  Form  614 

Unless  stated  otherwise,  references  in  this  subsection 
will  come  from  a  1989  report  entitled  Anal ysis  of.  Initial 
Spares  Support  Lists  ( ISSL) .  This  report  was  written  by  a 
team  headed  by  Captain  Steve  Reynolds. 

Developer( s) ;  Unidentified.  Policy  for  initial 
requirements  determination  is  made  by  HQ  AFLC/MMMIE.  The 
AFLC  Form  614s  have,  historically,  been  manually  prepared. 
This  process  is  being  automated  by  XRII  on  a  system  known  as 
the  Initial  Requirements  Determination  (IRD)  system.  The 
IRD  is  currently  in  system  validation  test  and  should  be  on 
line  by  the  end  of  summer  1991  (Horner,  1991). 

Model  Purpose :  The  AFLC  Form  614  computations  are  the 
"initial  spares"  equivalent  of  the  those  performed  by  D041 
to  determine  replenishment  spares  buy  requirements.  Unlike 
D041,  however,  the  AFLC  Form  614  process  has  not  been  used 
in  the  budget  development  process.  Initial  spares  budgets 
are  developed  using  a  factor  based  approach  that  relates 
spares  requirements  to  aircraft  flyaway  costs  (Rexroad, 
Tillia,  and  Tritle,  1990:1).  This  approach  has  been  used 


because  the  numerous  data  inputs  required  in  the  AFLC  Form 
614  process  are  typically  not  available  until  completion  of 
the  full-scale  development  life  cycle  phase  (TASC,  1989 :Ch 
5,  55).  Budget  wedges  must  be  estimated  prior  to  this  time 
and  so  HQ  AFLC/FMBSR,  the  agency  responsible  for  the  initial 
spares  budget,  has  fallen  back  on  the  factor  based  approach. 

The  factor  based  method  has  been  criticized  for  its 
lack  of  lack  of  insight  into  underlying  causal  relationships 
and  alternative  cost  models  are  being  evaluated  as 
candidates  to  replace  it.  The  Logistics  .Support  Cost  (LSC) 
Model  (the  next  model  evaluated  in  this  appendix)  has  been 
recommended  by  two  studies  for  this  purpose  (Dement,  1990; 
Rexroad,  Tillia,  and  Tritle,  1990).  The  LSC  model,  however, 
presents  the  same  data  input  scarcity  problem  which  has 
prevented  the  AFLC  Form  614  process  from  doing  the  job  all 
along.  Although  the  AFLC  Form  614  process  has  not  been  used 
as  a  cost  estimating  tool  in  the  past,  it  is  included  in 
this  appendix  because  it  seems  that  the  sairie  work  around 
being  suggested  for  the  LSC  data  problem  (i.e,,  using  data 
from  analogous  systems  (Dement,  1990:Ch  2))  would  work  for 
it  also. 

Model  Algorithm;  The  AFLC  Form  614  computations  are 
similar  in  nature  to  the  D041  computations.  As  stated  in 
the  D041  analysis,  the  provisioning  factors  used  in  the  AFLC 
Form  614  computations  are  estimates  based  upon  contractor 
inputs  and  the  expertise  of  the  equipment  specialist(s) 
making  the  estimate.  This  contrasts  with  the  D041  factors 
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which  may  be  adjusted  based  upon  observations  of  the  items' 
performanct  during  the  initial  spare  coverage  period  (an 
average  of  two  years)  (15).  Figure  15  provides  examples  of 
the  AFLC  Form  614  requirements  computations. 

Data  Inputs  and  Sources :  The  following  provisioning 
factors  are  provided  by  the  equipment  specialist( ’)  during 
^  the  provisioning  process: 

Maintenance  Factor 

Overhaul  Replacement  Percent 

Base  Condemnation  Percent 

Depot  Condemnation  Percent 

Not  Reparable  This  Station  (NRTS)  Percent 

In  addition  to  these  factors,  information  concerning 
the  weapon  system's  operational  pregram  is  collected  by  the 
System  Program  Manager  on  checklists  such  the  f^FLCR  57-27 
(18).  Together,  these  data  provide  the  inputs  required  to 
compute  the  APLC  Fcrm  614  initial  spares  requirements. 

Assessment :  Besides  the  previously  mentioned  data 

availability  problems,  there  are  two  important  deficiencies 
inherent  in  the  AFLC  Form  614  process  that  were  identified 
in  Capt  Reynold's  .CSSL  study. 

V 

The  first  problem  is  th?t  the  614  worksheets  do  not 
maximize  the  repair  capability  for  a  '.'i'.'tin  budget 
constraint.  The  reason  this  is  so  is  that  no  consideration 
is  given  to  the  LRU/SRU  indenture  relationships  (41).  It 
is  probable,  therefore,  that  a  better  mix  of  LRUs  and  SRUs 
could  provide  the  same  amount  of  repair  capability  at  a 
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Figure  15.  Example  APFiC  Form  614  Requirements  Computations 

(AFLCR  51~n ,  1986;  5-6) 


cheaper  price;  or,  using  the  same  budget,  a  greater  repair 
capability  coui d  be  obtained. 

The  second  problem  is  that  the  614  worksheets  have  "no 
direct  linkage  to  readiness  and  availability  objo^/cive"  as 
required  b”  DoD  mandate  (DODI  4140 . 42 )  ( 43.) .  The  614 
worksheet  computations  attempt  to  fill  backorders  for  items 
duriri  the  initial  spares  coverage  period,  but  there  is  no 
direct  link  made  between  filling  backorders  and  nesting  a 
readiness  objective.  Not  all  parts  are  critica.  for  an 
aircraft  to  be  "ready"  or  "available"  to  perform  its 
mission. 

Logistics  Support  Cost  ( LSC)  Model 

Unless  stated  otherwise,  references  in  this  subsection 
will  come  from  an  independent  validation  of  the  LSC  Model 
(version  2.0)  performed  by  Management  Consulting  &  Research, 
Inc.  (MCR) .  The  final  report,  written  by  Areve  B. 

Alexander,  Lori  E.  Brookey,  Robert  J.  Erhart,  Sarah  J. 
Fulton,  Dr.  Jerry  D.  Hofmann  and  Michael  D.  Shutak,  is  dated 
15  Kay  1990. 

Developer(s) :  Original  version  (1.0)  was  developed  in 
1975  by  the  Air  Force  Air  Logistics  Division  (AFALD)  to  run 
on  a  mainframe  computer  (II-l).  The  current  personal 
computer  based  version  (2.2)  was  released  in  1991  and 
developed  by  HQ  AFLC/FMC  (then  ACC)  (Passage,  1991). 

Model  Purpose ;  The  current  LSC  model  can  be  used  to 
provide  estimates  for  the  following  life  cycle  costs:  1) 
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initial  spares  (including  base  and  depot  safety  stock),  2) 
replenishment  spares,  3)  depot  maintenance,  4)  second 
destination  transportation,  and  5)  Repair  Support  Division 
(RSD)  Stock  Fund  (Passage,  1991). 

The  LSC  model  is  accepted  as  a  valid  estimating  model 
by  both  the  AF  and  OSD  Cost  Analysis  Improvement  Groups 
(CAIG).  Among  its  recent  applications,  the  LSC  model  was 
used  to  perform  independent  cost  analyses  for  the  Advanced 
Tactical  Fighter  (ATF),  C-17,  Global  Positioning  System,  and 
Joint  Stars  program  offices  (Passage,  1991). 

Model  Algorithm;  The  I..iC  model  performs  computations 
similar  to  those  used  by  3041  and  the  AFLC  Form  614  process. 
It  provides  cost  estimates  for  individual  reparable  items 
(LRU  or  SRU)  and  then  aggregates  these  costs  to  the 
subsystem  and  system  level  (1X1-2). 

Numerous  assumptions  are  incorporated  into  the  LSC 
model.  A  selection  of  these  assumptions  include: 


1)  "The  logistics  support  processes  and  costs  of  an 
item  are  mathematically  independent  of  those  of 
other  items,  even  items  physically  related  to  it" 
(III-2). 

2)  "The  support  cost  calculation  of  one  item  does  not 
affect  the  support  costs  calculation  for  any  other 
item"  (III-2). 

3)  "The  model  assumes  the  standard  USAF  maintenance 
concepts  for  the  repair  of  rvparable  components 
from  aircraft  apply"  (III-3).  For  this  reason,  the 
model  can  handle  three  level  or  two  level 
maintenance  scenarios. 

4)  A  Poisson  probability  function  is  used  in  the 
modeling  of  queues  (III-8). 
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5)  Little's  equation  is  used  to  calculate  the  number 
of  items  repaired  monthly  at  the  intermediate  level 
(III-7). 

6)  The  interiAediale  level  repair  process  meets  the 
following  requirements: 

a.  The  number  of  arrivals  per  unit  of  time  is 
described  by  a  Poisson  distribution. 

b.  Customers  are  served  in  order  of  arrival. 

-  c.  There  is  one  server  or  serving  facility. 

d.  The  mean  arrival  rate  is  less  than  the  mean 
service  rate. 

'  e.  The  waitinc  space  for  customers  is  infinite; 

that  is,  nc  customers  are  turned  away  or  leave 
of  their  ovn  accord,  due  to  limited  waiting 
space  or  slow  service  times. 

f.  The  population  of  customers  is  finite. 

g.  Service  times  are  describe  by  a  deterministic 
distribution  function.  (III-9) 

Figure  16  provides  examples  of  the  computations  used  by 
the  LSC  model . 

Data  Inputs  and  Sources :  The  LSC  model  contains  five 
data  input  files:  1)  system;  2)  hardware;  3)  cost;  4) 
support  equipment;  and  5)  SRO  factor  file.  Tables  18 
through  20  identify  required  data  inputs  and  data  sources 
for  three  of  these  files  to  illustrate  the  large  amount  of 
data  required  in  the  LSC  model  [the  system  file,  not  shown 
belovJ,  contains  another  29  data  inputs].  It  is  the  large 
number  of  data  inpuns  wlich  prohibits  this  type  of  model 
from  being  used  in  the  early  acquisition  phase  without 
resorting  to  analogous  systems  data  or  design  goals  to 
complete  the  data  files. 
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Figure  16.  Example  LSC  Model  Requirements  Computations 
(Passage,  1991) 


Table  18 


LSC  Hardware  Pile  Inputs  &  Sources 


DATA  INPUT 

SOURCE  1 

#  of  subsystems 

SPO/SPM 

WUC  identification 

SPO/SPM/ALD  i 

System  name 

SPO/SPM/ ALD  ! 

#  of  LRU/SRU  records 

SPO/SPM/ALD 

1  Design  or  target  mean  time  to 

1  repair  at  the  organizational 
level 

SPO/SPM/ALD 

Design  of  target  mean  time 
between  removals 

SPO/SPM/ALD 

Engineering  change  order 
indicator 

LSC  USER  I 

WUC  identification  of  LRU/SRU 

SPO/SPM/ALD 

Description  or  r.ame  of 

LRU/SRU 

SPO/SPM/ALD 

Stocklisted  ^tem  indicator 

SPO/SPM/ALD 

1  Quantity  per  application 

SPO/SPM/ALD  1 

1  Expandability, 

1  recoverability,  reparability 

1  category  (ERRC)  record  to  be 

1  read  from  System  File 

LSC  USER 

LRU/SRU  mean  time  between 
removal 

SPO/SPM/ALD 

1  LRU/SRU  mean  time  between 

1  demand 

SPO/SPM/ALD 

1  Average  #  of  hours  for  depot 

1  to  repair  an  SRU  from  this 

1  LRU 

ALC/SPO/SPM/ALD 
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Table  18  (Continued) 

LSC  Hardware  File  Inputs  &  Sources  (Passage,  1990) 


!  DATA  INPUT 

SOURCE 

Not  reparable  this  station  % 

SPO/SPM/ALD 

Base  condemnation  rate 

SPO/SPM/ALD 

Depot  condemnation  rate 

ALC/SPO/SPM/ALD 

Average  #  of  labor  hours  to 
repair  this  item  at  the  depot 

ALC/ SPO/SPM/ALD 

Labor  rate  for  Interim 
Contractor  Support  (ICS)  or 

RIW  repair 

SPO/SPM/AFLCP  173-10 

Fraction  of  time  this  LRU  is 
repaired  at  base  by  removal 
of  an  SRU  and  sending  an  SRU 
to  depot 

ALC/SPO/SPM 

LRU/ SRU  weight  (in  lbs) 

SPO/SPM/ALD 

SRU  indicator:  1  =  LRU  with 
unknown  SRUs;  2  =  LRU  with 

SRU  data  to  follow;  3  =  item 
is  an  SRU 

SPO/SPM/ALD 

Utilization  Factor  -  ratio  of 
operating  hours  to  flying 
hours 

SPO/SPM/ALD 

User  defined  category 
indicator 

LSC  USER 

Derating  index  indicator 

LSC  USER 
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Table  19 


LSC  Support  Equipment  File  Inputs  &  Sources  (Passage,  1990) 


DATA  INPUT 

i  1  1  1  II  1  1 . . . 

SOURCE 

Index  value  identifying  type 
of  support  equipment  (SE) 

SPO/SPM/ALD 

#  of  years  of  SE  costs 

LSC  USER 

Cost  of  SE  inherited  from 
another  Program 

SPO/SPM 

I  Cost  of  SE  developed  for  this 

1  Program 

SPO/COST  TEAM 

Table  20 

LSC  Cost  File  Inputs  &  Sources  (Passage,  1990) 


DATA  INPUT 

SOURCE 

5  digit  Work  Unit  Code  (WUC) 
matching  Hardware  File  WUC 
for  same  item 

SPO/SPM/ALD 

#  of  cost  fields  on  this  line 

LSC  USER 

Lot  average  unit  cost  for 
each  year  specified  by  second 

1  input  above 

SPO  COST  TEAM 

Assessment :  As  previously  stated,  the  LSC  model  has 

been  approved  for  use  by  both  the  AF  and  OSD  CAIGs  and  it 
was  independently  validated  by  MCR.  The  LSC  was  one  of  the 
two  models  chosen  (out  of  a  group  of  six  alternatives)  by  an 
Initial  Spares  Working  Group  as  an  alternative  to  the 
current  factor  based  approach  to  developing  initial  spares 
budgets  (Rexroad,  Tillia,  and  Tritle,  1990).  Another  study 
performed  by  Capt  Anne  Dement,  an  ALD  operations  research 
analyst,  attempted  to  validate  the  use  of  a  demand  based 
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budget  estimating  model  (i.e.,  the  LSC  model)  during  the 
early  acquisition  pi.  ses  by  using  it  to  estimate  initial 
spares  costs  for  the  Advanced  Tactical  Fighter.  According 
to  the  report's  Executive  Summary,  "the  test  showed  the 
approach  was  executable,  provided  a  reasonable  estimate,  and 
offered  several  significant  advantages  over  current 
estimating  techniques"  (Dement,  1990). 

Even  with  all  of  these  factors  in  its  favor,  the  LSC 
model  is  still  not  without  its  shortcomings.  Like  the  AFLC 
Form  614  process,  the  LSC  model  has  no  marginal  analysis 
capability  to  optimize  the  LRU/SRU  mix  for  a  given  budget 
constraint.  The  model  also  doesn’t  include  spares  required 
for  programmed  depot  maintenance  and  component  overhaul  in 
its  calculations  (Dement,  1990: Append  1,  Sec  2.2.2). 

Finally,  the  out  of  production  and  demand  volatility  factors 
used  in  the  model  need  additional  analysis  to  validate  them. 
In  both  cases  these  factors  make  significant  contribution  to 
the  overall  spares  requirements  and  yet  little  research  has 
been  conducted  to  validate  their  default  values  [see  Chapter 
two  of  this  thesis  for  a  description  of  the  demand 
volatility  factor  history]. 

Mod-METRIC 

Developer(s) :  John  A.  Muckstadt  developed  Mod-METRIC 

based  upon  the  earlier  Multi-Echelon  Technique  for 
Recoverable  Item  Control  (METRIC)  developed  by  Craig  C. 
Sherbrooke  of  the  RAND  Corporation  (Muckstaadt,  1973:472). 
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Model  Purpose;  While  METRIC  was  designed  for 
determining  both  the  requirements  and  distribution  of 
recoverable  spares  in  a  two-echelon  (base  and  depot) 
inventory  system  (Muckstadt,  1973:472),  Mod-METRIC  also 
’’permits  two  levels  of  parts  to  be  considered,  an  assembly 
and  its  components"  (Muckstadt,  1973:474).  In  other  words, 
Mod-METRIC  uses  marginal  analysis  techniques  to  identify  the 
optimum  mix  of  LRUs  and  SRUs  for  a  given  budget  constraint. 

Current  Mod-METRIC  applications  include  "initial 
provisioning,  engine  requirement  computations  and 
redistribution  of  spares"  (AFLCP  57-13,  1987:3). 

Model  Algorithm;  Keeping  weapon  systems  in  serviceable 
condition  is  the  goal  of  the  Air  Force  maintenance  process 
described  in  this  paragraph  (AFLCP  57-13,  1987:3-4).  Modern 
weapon  systems  have  modular  designs  to  allow  failed  LRUs  or 
SRUs  to  simply  be  removed  and  replaced  from  the  base  stock 
of  serviceable  spares.  The  stock  is  replenished  when  the 
base  repairs  the  failed  item  and  places  it  in  the  stock.  If 
the  base  cannot  repair  the  item,  it  is  sent  to  the  depot  for 
repair  and  the  base  requests  that  its  serviceable  spares 
stock  be  resupplied  from  the  depot  stock.  It  is  clear  that 
the  stock  levels  of  serviceable  items  impact  the  performance 
of  this  system.  If  demanded  items  cannot  be  supplied  from 
the  base  stock,  the  aircraft  goes  into  not  mission  capable 
(NMC)  status  until  the  item  is:  1)  repaired  at  the  base 
level,  2)  repaired  at  the  depot  level,  or  3)  obtained  from 
the  depot  stock  (if  available).  When  items  demanded  at  the 
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base  level  cannot  be  supplied  from  base  stock,  the  items  are 
considered  on  backorder.  One  backorder  day  equates  to  one 
item  on  backorder  for  one  day  (AFLCP  57-13,  1987:8). 

Both  METRIC  and  MOD-METRIC  model  this  maintenance 
process.  The  objective  function  for  METRIC  is  to  determine 
the  amount  of  LRU  stock  and  where  to  place  it  (base  or 
depot)  that  will  minimize  the  number  of  backorder  days  for  a 
given  budget  constraint  (APLCP  57-13,1987:10).  The 
objective  function  for  MOD-METRIC  focuses  on  a  single  LRU 
and  determines  the  optimum  mix  of  the  LRU  and  its  associated 
SRUs  that  will  minimize  the  number  of  backorder  days  for  the 
LRU  for  a  given  budget  constraint  [Another  METRIC  based 
model  known  as  COMBINE  supplements  t.ie  MOD-METRIC  model  to 
solve  problems  with  many  LRU-SRU  groups  considered 
simultaneously]  (AFLCP  57-13,  1987:1)). 

Both  METRIC  and  Mod-METRIC  have  the  same  assumptions 
(AFLCP  57-13,1987:8): 

a.  No  lateral  resupply  between  bases. 

b.  No  batching  of  items  before  repair  is  started 
on  an  item  (infinite  channel  queuing  assumption). 

c.  The  level  at  which  repair  is  performed  depends 
only  on  the  complexity  of  the  repair,  not  on 
existing  workload. 

d.  Repair  times  are  statistically  independent. 

e.  A  stationary  compound  Poisson  probability 
distribution  describes  the  demand  process  for 
each  item. 

f.  Simultaneous  failures  of  SRUs  do  not  occur. 

g.  A  failure  of  one  type  of  item  is  statistically 
independent  of  those  that  occur  for  any  other  type  of 
item. 

Both  the  METRIC  and  MOD-METRIC  models  employ  operations 
science  techniques  such  as  Lagrangian  multipliers  and 
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Fibonnacci  searches.  The  scope  of  this  paper  limits  the 
following  algorithm  discussion  (AFLCP  57-13,  1987:9-10)  to  a 
more  general  level.  As  previously  stated,  METRIC  determines 
optimum  LRU  stock  levels  for  a  given  budget  constraint. 
Because  partial  LRUs  are  not  considered,  the  cost  of  the 
stock  level  will  probably  not  equal  the  budget  constraint 
exactly.  The  Lagrangian  multipliers  are  part  of  a  marginal 
analysis  process  that  evaluates  the  reduction  in  backorders 
per  additional  dollar  invested  in  the  various  LRUs  which 
compose  the  weapon  system  and  provides  a  solution  very  close 
to  the  budget  constraint.  The  model  then  increments  the 
budget  and  determines  the  optimum  stock  level  for  the  new 
budget  constraint.  This  allows  one  to  plot  backorder 
performance  versus  cost. 

The  MOD-METRIC  model  has  the  added  complexity  of 
evaluating  the  optimum  mix  of  an  LRU  and  its  SRUs.  The 
available  budget  is  first  allocated  between  purchasing  whole 
LRUs  and  the  associated  SRUs  using  the  Fibonnacci  search. 
After  a  portion  of  the  budget  has  been  allocated  to  SRUs, 
the  same  marginal  analysis  employed  by  METRIC  is  used  to 
determine  the  optimum  number  of  the  various  SRUs  which 
compose  the  LRU  and  to  distribute  these  SRUs  between  the 
bases  and  the  depot.  The  budget  is  then  incremented  and  the 
process  is  repeated  for  the  new  budget  constraint.  Again, 
performance  versus  cost  can  be  plotted. 

Data  Inputs :  Data  files  for  running  mod-METRIC  can  be 
created  at  HQ  AFLC  using  CREATE  (a  Honeywell  6000  series 
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computer).  Mod-METRIC  may  be  run  from  other  systems  given 
that  specific  formats  (addressed  in  'hapter  eight  of  AFLCP 
57-13)  are  adhered  to  in  developing  he  data  files.  The 
following  data  inputs  are  required  (ivFLCP  57-13,  1987:15): 

Flying  hour  program  data: 

1)  Number  of  bases. 

2)  Flying  hours  per  month  )er  base. 

3)  Order  and  shipping  time  per  base. 


Recoverable  item  data: 

1)  Identification. 

2)  Unit  cost. 

3)  Mean  time  between  deman  I . 

4)  Not  repairable  this  sta:ion  percentage. 

5)  Condemnation  percentage. 

6)  Quantity  per  next  higher  application. 

7)  Base  repair  time. 

8)  Depot  repair  time. 

9)  Procurement  lead  time. 

10) .  Item-dependent  order  and  ship  time. 

Computer  program  control  data:  This  data  category 
refers  to  information  provided  by  tha  user  when  prompted  by 
a  series  of  questions  from  an  interactive  preprocessor 
called  BUILD.  The  answers  to  these  juestions  (e.g. ,  user- 
id,  target  budget  value,  size  of  increment  between 
successive  budgets  evaluated,  etc.)  are  required  for  the 
program  to  know  the  specifics  of  what  type  of  output  the 
user  is  seeking  (AFLCP  57-13,  1987:15-17). 

Assessment :  The  F-15  program  was  one  of  the  first 
weapon  systems  to  employ  the  MOD-METIIC  model.  Previous 
weapon  systems  made  use  of  the  methodology  prescribed  in 
AFLCR  57-27  (still  an  accepted  approach).  This  methodology 
"was  written  and  published  when  maximum-base-self - 
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sufficiency  was  an  accepted  maintenance  concept"  (Huff, 
1979:9).  This  concept  does  not  allow  for  optimization  of 
resources  because  individual  bases,  using  this  approach, 
tend  to  purchase  more  LRUs  and  less  SRUs  to  prevent  LRU 
backorders.  More  repair  parts  are  required  at  each  base  in 
order  for  them  to  be  self-sufficient.  The  MOD-METRIC  model 
is  credited  as  one  of  the  factors  behind  the  F-15  program's 
ability  to  complete  initial  spares  provisioning  within 
budget  (Huff,  1979:10-11). 

The  Mod-METRIC  model  is  not  without  limitations, 
however.  Because  the  Mod-METRIC  model  assumes  that  demand 
is  described  by  a  stationary  compound  Poisson  probability 
distribution  and  that  each  item  is  equally  critical,  critics 
argue  that  the  use  of  backorders  to  approximate  aircraft 
availability  is  not  adequate.  The  Dyna-METRIC  model  was 
developed  after  Mod-METRIC  to  accommodate  for  the  dynamic 
requirements  process  (e.g.  surge  requirements  in  a  war-time 
scenario)  and  the  differing  criticality  of  different  spares 
(Mills,  1985:11). 

Dvna-METRIC 

Unless  stated  otherwise,  all  references  from  this 
subsection  will  come  from  a  1988  RAND  Corporation  report 
entitled,  Dyna-METRIC  Version  ^  Model ing  Worldwide  Logistics 
Support  of  Aircraft  Components .  The  authors  were  Karen  E. 
Isaacson,  Patricia  Boren,  Christopher  L.  Tsai  and  Raymond 
Pyles . 


211 


Devel oper (s ) :  The  Dyna-METRIC  moc’el  has  evolved  as  the 


result  of  several  RAND  Corporation  projects  dating  back  to 
the  1970's.  R.  J.  Hillestad  and  M.  J.  Carrillo  are  credited 
with  the  "theoretical  development  of  the  dynamic  queuing 
equations  that  form  the  heart  of  Dyna-METRIC  Hillestad, 
1982:5).  The  Headquarters  Air  Force  Logistics  Command 
Management  Sciences  Division  (HQ  AFLC/XPS)  is  responsible 
for  the  current  version  of  the  model  (iii). 

Model  Purpose :  Dyna-METRIC  is  a  readiness,  or 
availability,  based  model  that  "relates  logistics  resources 
and  policies  to  wartime  readiness"  (V).  It  was  developed 
for  logisticians  to  assess  the  impact  of  "wartime  dynamics 
and  repair  constraints  and  provides  operational  performance 
measures,  problem  detection,  and  spares  requirements"  (V). 
Dyna-METRIC  will  soon  replace  Mod-METRIC  as  the  model  used 
to  determine  whole  engine  and  engine  module  requirements 
(both  wartime  and  peacetime  requirements).  Additionally, 
code  is  currently  being  written  to  incorporate  Dyna-METRIC 
into  the  Logistics  Support  Cost  (LSC)  model.  This  will 
change  the  LSC  from  a  pure  demand  based  approach  to  one  that 
uses  marginal  analysis  to  maximize  improvements  in 
availability  per  dollar  spent  (Niklas,  1991).  Dyna-METRIC 
allows  one  to  model  three  echelons  of  maintenance  repair  and 
three  indenture  levels  of  components  (V). 

Model  Algorithm:  Dyna-METRIC  allov7S  one  to  predict  the 
number  of  aircraft  components  that  are  flowing  from  point  to 
point  in  the  repair  process  (i.e.,  through  repair 
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"pipelines").  Central  ',o  these  predictions  is  the  delay  or 
processing  time  these  components  must  spend  in  each  pipeline 
segment.  "The  expected  number  of  components  in  each 
segment,  then,  depends  on  the  rate  at  which  demands  occur 
and  the  time  the  components  spend  in  each  segment"  (VI). 

Dyna-METRIC  uses  the  user-supplied  pipeline  variance  to 
mean  ratio  (VTMR)  to  determine  what  type  of  distribution  to 
use  to  predict  the  number  of  components  within  a  pipeline 
segment.  A  VTMR  of  less  than  one  corresponds  to  a  binomial 
distribution.  A  VTMR  of  one  suggests  a  poisson 
distribution,  and  a  VTMR  of  greater  than  one  suggest  a 
negative  binomial  distribution.  These  distributions  are 
used  by  the  model  to  "recommend  st..  «;  levels  for  components" 
and  "to  predict  performance  at  each  base  for  each  time  of 
analysis"  (92). 

Unlike  Mod-METRIC,  Dyna-METRIC  also  allows  one  to 
consider  policies  of  full  and  partial  cannibalization  of 
aircraft.  The  full  cannibalization  routine  involves  two 
assumptions.  First,  it  assumes  that  all  the  aircraft  at  a 
base  look  alike  (i.e.,  composed  of  the  same  LRUs)  and  will 
lead  to  overly  optimistic  results  if  this  assumption  is 
violated.  The  second  assumption  is  that  "cannibalization 
can  be  done  instantly  and  without  consuming  resources"  (95). 
When  either  of  the  above  assumptions  are  broken,  it  is 
recommended  that,  if  one  wants  to  include  cannibalization  in 
one's  model,  a  partial  cannibalization  policy  is  modeled  in 
lieu  of  the.  full  cannibalization  routine  (94-95). 
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It  is  beyond  the  scope  of  this  appendix  to  introduce 
the  numerous  mathematical  aquations  utilized  by  Dyna-METRIC. 
Interested  readers  should  consult  the  reference  listed  at 
the  beginning  of  this  model's  analysis. 

Data  Inputs :  The  Dyna-METRIC  model  requires  numerous 
user  inputs.  The  following  is  a  partial  list  of  these 
inputs : 

*  Expected  pipeline  size  (i.e.,  component  quantity) 
at  the  location  being  analyzed 

*  Pipeline  Variance  to  Mean  Ratio  (VTMR) 

*  Component  stock  level 

*  Number  of  aircraft  at  the  base,  after  attrition 

*  Number  of  different  types  of  LRU  of  which  the 
aircraft  are  constructed 

*  Quantity  per  application  cf  LRU  i 

*  Quantity  per  application  of  LRU  i  that  must  be 

operational  if  the  aircraft  is  fully  mission 
capable  (PMC) 

*  LRU  i  stock  level  at  base  (92-93) 


Assessment ;  Dyna-METRIC' s  ability  to  predict  logistics 
requirements  under  dynamic  conditions  has  made  it  a  valuable 
tool.  Early  uses  of  the  model  are  numerous.  Ogden  Air 
Logistics  Center  u.ced  the  model  to  study  F-4  and  F-16 
aircraft  readiness  and  supportability .  Headquarters  Air 
Force  Logistics  Command  used  it  to  study  FlviO  engine 
requirements,  and  the  Tactical  Air  Command  has  used  it  to 
analyze  the  impact  of  different  repair  and  supply  strategies 
on  F-15  readiness  and  deployability  (Hillestad,  1982:iv). 

Dyna-METRIC  is  not  without  it.<3  limitations,  however. 
Although  the  model  allows  one  to  model  multiple  bases,  this 
capability  is  weakened  because  the  model  can  not  handle 
multiple  types  of  collocated  aircraft.  Additionally, 
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lateral  supply  and  flight  line  constraints  are  also  "not 
explicitly  modeled"  (V). 

A  1988  RAND  Corporation  report  questions  the  ability  of 
models  such  as  Dyna-METRIC  to  model  the  "failure  process  and 
the  resulting  series  of  random  demands  on  supply  and 
maintenance"  (Crawford,  1988:V).  Specifically,  those  models 
which  assume  that  component  removals  follow  a  Poisson 
arrival  process  "assume  the  flow  of  broken  parts  {'demand 
process')  into  the  repair  facility  will  be  random,  but  with 
a  certain  mean  and  a  certain  degree  of  randomness  or 
'irregularity'  (Crawford,  1988:4).  The  variance-to-mean 
ratio  (VTMR)  is  a  measure  of  this  variability  and  thus  the 
unpredictability  of  the  demand  process.  Crawford's  findings 
show  that  actual  VTMRs  experienced  are  significantly  greater 
than  those  assumed  by  the  models  (such  as  Dyna-METRIC) 
(Crawford,  1988:1).  When  this  happens,  "the  part  will 
arrive  at  the  repair  process  in  what  appear  to  be  large 
random  clusters  instead  of  random,  but  more  evenly  spaced, 
intervals"  (4).  These  results,  according  to  Crawford, 
reduce  the  confidence  one  can  place  in  these  models  and  may 
have  a  "damaging  effect  on  aircraft  availability  and  wartime 
readiness"  (Crawford,  1988 :vi), 

Dyna-METRIC  also  fails  to  identify  the  least  cost  mix 
of  LRUs  and  SRUs  needed  to  satisfy  availability  goals.  The 
Aircraft  Sustainability  Model  (ASM))  was  developed  by  the 
Logistics  Management  Institute  (LMI)  to  correct  this  problem 
(Oster,  Sakulich,  and  Stone,  1989:2). 
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Appendix  D.  Condemnation;;  CER  Database 


The  data  in  Table  21  represents  the  entire  database 
used  in  the  development  of  condemnation  CTRs  for  this 
thesis.  The  data  comes  from  numerous  sources.  The  annual 
condemnation  costs,  number  of  aircraft,  and  annual  flying 
hours  were  all  obtained  from  the  Weapon  System  Cost 
Retrieval  System  (H036C) . 

The  annual  sorties  and  annual  landings  were  obtained 
from  v/hat  is  currently  referred  to  as  Acquisition  Logistics 
Division  (ALD)  Pamphlet  800-4,  Acquisition  Management 
Aircraft  Historical  Reliability  and  4aintainabil itv  Data . 
Because  ALD  has  been  known  by  different  names  through  the 
years,  the  different  volumes  of  800-4  have  different  titles. 
The  first  three  volumes  were  labeled  AFALD  Pamphlet  800-4. 
Sorties  and  landings  data  from  FY75  through  the  first  half 
of  FY78  was  taken  from  Volume  I.  Data  from  the  second  half 
of  FY78  through  FY80  came  from  Volume  II,  and  data  from  FY81 
through  the  first  half  of  FY83  came  from  Volume  III.  The 
next  two  volumes  were  known  as  the  Air  Force  Acquisition 
Logistics  Center  (AFALC)  Pamphlet  800-4.  Data  from  the 
second  half  of  FY83  through  FY85  came  from  Volume  IV,  and 
data  from  FY86  through  FY87  came  from  Volume  V.  Finally, 
data  from  FY88  to  FY89  came  from  Volume  VI,  known  as  ALD 
Pamphlet  800-4. 

The  remaining  physical  and  performance  characteristics 
data  was  taken  from  various  addenda  to  AFG  2,  Standard 


Aircraft  Characteristics.  Data  for  the  F-4D  and  F-4E  came 
from  Addendum  55,  Volume  1.  Data  for  the  C-5A  came  from 
Addendum  53,  Volume  2.  Data  for  the  C-130B  and  T-38A  came 
from  Addendum  54,  Volume  2.  Data  for  the  F-.16A  came  from 
Addendum  58,  Volume  1.  Data  for  the  A-7D,  A-7K,  KC-135A. 
and  A-lOA  came  from  Addendum  59,  Volume  1.  Data  for  the  T 
37B,  C-130E,  and  C-141B  came  from  Addendum  56,  Volume  2. 

Data  for  the  F-15A  and  F-15B  came  from  September  1989 
Standard  Aircraft  Characteristics  pamphlets  (no  AFG 
references  were  found  on  these  pamphlets).  Finally,  the  B- 
52G,  B-52H,  F-lllD,  and  FB-lllA  data  wa.s  taken  from  Addendum 
60,  Volume  1. 
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Table  21 


Condemnations  CER  Database 


ANNUAL 

# 

ANNUAL 

ANNUAL 

D 

U 

CONDEMN 

OF 

ANNUAL 

#  OF 

#  OF 

M 

EMPTY 

MDS 

FY 

COSTS 

A/C 

FLY  HRS 

SORTIES 

LANDINGS 

Y 

WEIGHT 

F15A 

85 

56,284,486 

321 

71,586 

0 

26,749 

F15A 

86 

46,541,266 

318 

75,559 

0 

26,749 

F15A 

87 

46,559,725 

317 

74,363 

0 

26,749 

F15A 

88 

46,305,459 

314 

69,722 

0 

26,749 

F15A 

89 

39,983,476 

313 

71,684 

0 

26,749 

F15B 

85 

9,949,506 

54 

12,691 

0 

26,832 

F15B 

86 

8,634,666 

54 

14,114 

0 

26,832 

F15B 

87 

9,087,565 

53 

14,327 

0 

26,832 

F15B 

88 

8,948,457 

54 

13,495 

0 

26,832 

F15B 

89 

7,364,837 

53 

13,382 

0 

26,832 

FI  6  A 

85 

62,780,304 

627 

171,748 

125,503 

127,526 

0 

15,306 

F16A 

86 

60,804,303 

619 

168,671 

126,265 

127,488 

0 

15,306 

F16A 

87 

67,173,261 

615 

175,137 

134,365 

138,092 

0 

15,306 

FI  6  A 

88 

61,454,897 

604 

167,914 

112,759 

113,563 

0 

15,306 

FI  6  A 

89 

55,416,270 

563 

165,706 

112,336 

113,021 

0 

15,306 

F4D 

76 

15,609,443 

494 

104,756 

68,020 

0 

28,873 

F4D 

77 

13,008,510 

472 

101,750 

70,210 

0 

28,873 

F4D 

78 

17,001,188 

467 

96,590 

72,091 

0 

28,873 

F4D 

79 

20,153,467 

456 

100,067 

77,301 

91,781 

0 

28,873 

F4D 

80 

22,737,395 

450 

89,927 

69,192 

86,888 

0 

28,873 

F4E 

77 

13,565,579 

697 

168,900 

121,195 

0 

30,328 

F4E 

78 

19,369,172 

671 

158,431 

119,474 

0 

30,328 

F4E 

79 

18,444,521 

626 

155,493 

119,391 

144,590 

0 

30,328 

F4E 

80 

19,210,958 

546 

129,388 

101,179 

114,328 

0 

30,328 

F4E 

81 

24,254,355 

499 

113,343 

89,376 

98,232 

0 

30,328 

A7D 

76 

13,370,606 

407 

82,159 

51,267 

0 

19,733 

A7D 

77 

9,251,047 

406 

109,914 

64,787 

0 

19,733 

A7D 

78 

14,151,911 

384 

101,421 

61,127 

0 

19,733 

A7D 

79 

13,002,152 

376 

94,101 

59,537 

59,903 

0 

19,733 

A7D 

80 

13,564,173 

371 

91,423 

59,985 

60,285 

0 

19,733 

A7K 

83 

1,251,303 

31 

8,292 

0 

21,300 

A7K 

84 

1,057,256 

31 

8,630 

0 

21,300 

A7K 

85 

1,179,063 

29 

8,122 

5,520 

5,533 

0 

21,300 

A7K 

86 

1,145,533 

30 

6,040 

4,159 

4,181 

0 

21,300 

A7K 

87 

919,806 

30 

6,583 

4,450 

4,464 

0 

21,300 

AlOA 

84 

17,683,981 

668 

226,171 

0 

21,541 

AlOA 

85 

18,505,945 

663 

222,569 

132,998 

133,776 

0 

21,541 

AlOA 

86 

22,482,695 

659 

219,958 

132,998 

133,314 

0 

21,541 

AlOA 

87 

26,952,091 

651 

224,177 

134,048 

134,057 

0 

21,541 

AlOA 

88 

22,279,258 

627 

211,892 

128,100 

128,119 

0 

21,541 

FlllD 

78 

13,729,867 

90 

15,140 

6,335 

0 

46,949 

FlllD 

79 

13,600,558 

85 

17,270 

7,416 

11,288 

0 

46,949 

1 

] 


-j 

Ij 

T 

3 

i 
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Table  21  (Continued) 
Condemnations  CER  Database 


ANNUAL 

# 

ANNUAL 

ANNUAL 

D 

U 

CONDEMN 

OF 

ANNUAL 

#  OF 

#  OF 

M 

EMPTY 

MDS 

FY 

COSTS 

A/C 

FLY  HRS 

SORTIES 

LANDINGS 

Y 

WEIGHT 

FlllD 

80 

15,719,180 

84 

17,828 

8,152 

12,108 

0 

46,949 

FlllD 

81 

14,234,373 

83 

17,906 

7,811 

11,609 

0 

46,949 

FlllD 

82 

13,848,110 

82 

16,933 

7,666 

11,678 

0 

45,949 

T37B 

76 

5,344,323 

749 

255,018 

208,570 

0 

4,067 

T37B 

77 

4,247,758 

680 

273,220 

219,115 

0 

4,067 

T37B 

78 

6,278,631 

697 

255,861 

94,961 

0 

4,067 

T37B 

79 

7,757,701 

668 

288,414 

229,489 

790,846 

0 

4,067 

T37B 

80 

5,521,700 

649 

288,141 

231,083 

798,415 

0 

4,067 

T38A 

75 

15,501,426 

990 

404,275 

338,665 

0 

7,410 

T38A 

76 

10,235,854 

984 

309,753 

247,519 

0 

7,410 

T38A 

77 

13,817,793 

916 

338,751 

278,209 

0 

7,410 

T38A 

78 

14,497,282 

819 

305,805 

119,751 

0 

7,410 

T38A 

79 

18,747,947 

777 

312,855 

257,550 

829,486 

0 

7,410 

B52G 

76 

35,158,761 

174 

69,194 

9,335 

1 

180,041 

B52G 

77 

49,605,497 

172 

65,512 

8,496 

1 

180,041 

B52G 

78 

52,700,602 

173 

64,269 

8,324 

1 

180,041 

B52G 

79 

56,680,050 

173 

65,013 

9,042 

25,617 

1 

180,041 

B52G 

80 

54,222,043 

173 

63,713 

9,705 

27,839 

1 

180,041 

B52H 

76 

16,499,476 

97 

36,635 

4,920 

1 

184,291 

B52H 

77 

20,263,183 

97 

36,346 

4,543 

1 

184,291 

B52H 

78 

21,980,774 

96 

36,513 

4,567 

1 

184,291 

B52H 

79 

22,330,298 

96 

36,691 

5,075 

15,055 

1 

184,291 

B52H 

80 

22,374,407 

9  j 

36,430 

5,103 

13,971 

1 

184,291 

C5A 

76 

37,813,318 

73 

36,710 

7,252 

1 

320,085 

C5A 

77 

34,824,717 

77 

48,590 

9,948 

1 

320,085 

C5A 

78 

41,647,986 

75 

48,282 

4,693 

1 

320,085 

C5A 

79 

44,176,326 

77 

48,657 

10,222 

31,690 

1 

320,085 

C5A 

80 

46,922,447 

77 

51,133 

10,570 

32,389 

1 

320,085 

C130B 

76 

3,968,334 

97 

37,811 

14,905 

1 

72,300 

C130B 

77 

2,893,091 

94 

36,585 

14,208 

1 

72,300 

C130B 

78 

2,738,158 

93 

37,836 

18,398 

1 

72,300 

C130B 

79 

2,781,287 

9'i 

36,923 

19,536 

50,361 

1 

72,300 

C130B 

80 

4,587,832 

9: 

37,621 

20,285 

52,571 

1 

72,300 

C130E 

76 

16,066,401 

338 

169,480 

74,174 

1 

73,804 

C130E 

77 

12,798,896 

309 

162,304 

66,747 

1 

73,804 

C130E 

78 

12,935,999 

280 

164,117 

90,682 

1 

73,804 

C130E 

79 

12,414,932 

27' 

161,280 

96,839 

225,448 

1 

73,804 

C130E 

80 

18,359,622 

27-5 

162,342 

99,644 

234,637 

1 

73,804 

C141B 

82 

50,958,201 

26) 

262,960 

73,405 

161,239 

1 

140,882 

C141B 

83 

55,749,449 

268 

289,119 

87,879 

177,036 

1 

140,882 

C141B 

84 

53,723,129 

268 

290,509 

1 

140,882 

C141B 

85 

39,329,435 

267 

290,142 

89,976 

184,982 

1 

140,882 

C141B 

86 

44,991,514 

266 

289,039 

88,977 

187,656 

1 

140,882 
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Table  21  (Continued) 
Condemnations  CER  Database 


ANNUAL 

# 

ANNUAL 

ANNUAL 

D 

U 

CONDEMN 

OF 

ANNUAL 

#  OF 

#  OF 

M 

EMPTY 

MDS 

FY 

COSTS 

A/C 

FLY  HRS 

SORTIES 

LANDINGS 

Y 

WEIGHT 

KC135A 

75 

57,757,551 

607 

220,148 

47,430 

1 

97,030 

KC135A 

76 

41,902,937 

606 

205,175 

39,807 

1 

97,030 

KC135A 

77 

60,962,617 

599 

189,271 

43,958 

1 

97,030 

KC135A 

78 

70,731,943 

598 

193,618 

47,549 

1 

97,030 

KC135A 

79 

81,852,238 

587 

193,084 

49,601 

157,466 

1 

97,030 

FBlllA 

76 

10,251,819 

74 

16,886 

3,892 

0 

47,481 

FBlllA 

77 

10,123,838 

70 

18,469 

4,870 

0 

47,481 

FBlllA 

78 

16,176,540 

68 

15,839 

4,566 

0 

47,481 

FBlllA 

79 

13,769,682 

67 

18,286 

5,416 

9,295 

0 

47,481 

FBlllA 

80 

19,071,741 

66 

17,443 

5,418 

10,491 

0 

47,481 

Table  21  (Continued) 
Condemnations  CER  Database 


THRUST 

E  LENGTH 

MAX 

MAX 

MAX 

PER 

N 

PLUS 

MAX 

TAKEOFF 

LOAD 

CLIMB  COMBAT 

MDS 

ENGINE 

G 

SPAN 

SPEED 

WEIGHT  FACTOR 

RATE  RADIUS 

Fi5A 

23,830 

2 

106.56 

1,309 

56,000 

7.33 

61,340 

515 

F15B 

23,830 

2 

106.56 

1,309 

56,000 

7.33 

59,930 

502 

F16A 

23,830 

1 

82.28 

1,181 

35,400 

9 

60,288 

693 

F4D 

17,000 

2 

96.6 

1,210 

59,483 

6.5 

55,600 

783 

F4E 

17,900 

2 

101.4 

1,245 

61,795 

7.75 

49,800 

741 

A7D 

14,250 

1 

84.8 

608 

39,325 

7 

8,000 

600 

A7K 

14,500 

1 

87.42 

569 

42,000 

7 

9,485 

282 

AlOA 

9,065 

2 

110 . 5 

362 

49,774 

7.33 

6,203 

351 

FlllD 

20,840 

2 

136.47 

1,262 

100,000 

7 

45,000 

1,270 

T37B 

1,025 

1 

63.1 

357 

6,800 

6.67 

3,600 

167 

T38A 

3,850 

2 

71.6 

709 

11,761 

7.33 

33,300 

305 

B52G 

13,750 

8 

346.9 

549 

488.000 

2 

8,243 

3,118 

B52H 

17,000 

8 

345.3 

547 

488,000 

2 

9,628 

3,747 

C5A 

40,805 

4 

470.5 

495 

769,000 

2.5 

5,580 

2,519 

C130B 

9,388 

4 

230.4 

330 

135,000 

3 

4,420 

1,549 

C130E 

9,388 

4 

230.4 

317 

175,000 

3 

4,060 

1,866 

C141B 

21,000 

4 

328.4 

493 

323,100 

2.5 

6,300 

2,366 

KC135A 

13,750 

4 

267.00 

527 

300,800 

2.00 

6,350 

1,613 

FBlllA 

20,350 

2 

145.54 

1,262 

119,243 

3.00 

33,800 
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Appendix  E.  Model  Validation  Database 


The  following  table  provides  the  data  used  in  both  the 
validation  and  sensitivity  tests.  Data  included  in  Table  22 
taken  from  the  same  sources  used  in  Table  21. 

Table  22 

Model  Validation  Database 


CONDEMN 

#  OF 

ANNUAL 

FLY  HRS 

#  OF 

MDS 

COSTS 

MDS 

FLY  HRS 

PER  AC 

SORTIES 

F16A 

27,537,555 

1983 

514 

118,018 

229.61 

85,379 

F4D 

24,151,890 

1981 

445 

89,844 

201.90 

68,869 

F4E 

24,254,355 

1981 

499 

113,343 

227.14 

89,376 

A7D 

20,075,237 

1981 

360 

83,120 

230.89 

56,073 

A7K 

1,560,822 

1988 

30 

7,381 

246.03 

4,874 

AlOA 

12,012,017 

1989 

586 

218,690 

373.19 

126,809 

FlllD 

14,831,043 

1983 

82 

18,368 

224.00 

8,406 

B52G 

43,909,467 

1981 

172 

63,959 

371.85 

9,465 

B52H 

20,415,995 

1981 

96 

37,873 

394.51 

5,226 

C5A 

71,036,363 

1981 

77 

52,160 

677.40 

10,373 

C130B 

4,698,837 

1981 

91 

36,443 

400.47 

20,310 

C130E 

18,837,210 

1981 

279 

168,526 

604.04 

99,436 

C141B 

51,264,650 

1987 

267 

284,065 

1,063.91 

87,102 

FBlllA 

13,471,819 

1981 

63 

17,332 

275.11 

5,513 
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Table  22  (Continned) 
Model  Validation  Database 


TAKEOFF 


FLY  HRS 

THRUST 

WEIGHT 

PER 

PER 

EMPTY 

MDS 

SORTY 

ENGINE 

WEIGHT 

F16A 

1.38 

23,830 

2.31 

F4D 

1.30 

17,000 

2.06 

F4E 

1.27 

17,900 

2.04 

A7D 

1.48 

14,250 

1.99 

A7K 

1.51 

14,500 

1.97 

AlOA 

1.72 

9,065 

2.31 

FlllD 

2.19 

20,840 

2.13 

B52G 

6.76 

13,750 

2.71 

B52H 

7.25 

17,000 

2.65 

C5A 

5.03 

40,805 

2.40 

C130B 

1.79 

9,388 

1.87 

C130E 

1.69 

9,388 

2.37 

C141B 

3.26 

21,000 

2.29 

FBlllA 

3.14 

20,350 

2.37 

/  MAX 


FLYAWAY 

MAX 

LOAD 

COST 

SPEED 

FACTOR 

11,646,586 

1,181 

9.00 

8,433,735 

1,210 

6.50 

10,140,562 

1,245 

7.75 

8,734,940 

608 

7,00 

16,767,068 

569 

7.00 

8,734,940 

362 

7.33 

39,859,438 

1,262 

7.00 

54,819,277 

549 

2.00 

60,240,964 

547 

2.00 

139,959,839 

495 

2.50 

18,172,691 

330 

3.00 

10,943,775 

317 

3.00 

34,437,751 

493 

2.50 

39,658,635 

1262 

3.00 
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Appendix  F.  Model  Sensitivity  Test  Results 


Table  23 

Model  Sensitivity  Test  Results 


"SORT"  Variable  Multiplied  by  1 . 2 


NEW 

NEW 

NEW 

EST./ 

NEW 

EST./ 

LINEAR 

ORIG. 

ARITHMETIC 

ORIG. 

MDS 

ESTIMATE 

EST. 

ESTIMATE 

EST. 

F16A 

42,377,500 

1.06 

41,424,889 

1.06 

F4D 

20,732,006 

1.11 

20,364,706 

1.11 

F4E 

24,314,651 

1.12 

23,999,556 

1.12 

A7D 

12,487,371 

1.15 

12,469,575 

1.15 

A7K 

2,622,908 

1.06 

3,264,162 

1.04 

AlOA 

34,227,437 

1.12 

31,527,170 

1.13 

FlllD 

17,116,714 

1.01 

17,239,688 

1.01 

B52G 

36,356,178 

1.01 

35,914,610 

1.01 

B52H 

36,263,284 

1.00 

35,856,568 

1.00 

C5A 

50,712,950 

1.01 

51,933,888 

1.01 

C130B 

(4,617,817) 

0.88 

(3,648,991) 

0.86 

C130E 

32,373,  571 

1.10 

29,873,545 

1.10 

C141B 

38,679,  137 

1.07 

37,494,591 

1.07 

FBlllA 

27,094,376 

1.01 

26,468,474 

1.01 

"SORT"  Variable 

Multiplied  by  .8 

NEW 

NEW 

NEW 

EST./ 

NEW 

EST./ 

LINEAR 

ORIG. 

ARITHMETIC 

ORIG. 

MDS 

ESTIMATE 

EST. 

ESTIMATE 

EST. 

F16A 

37,282,081 

0.94 

36,588,133 

0.94 

F4D 

16,621,904 

0.89 

16,463,249 

0.89 

F4E 

18,980,691 

0.88 

18,936,369 

0.88 

A7D 

9,140,934 

0.85 

9,293,016 

0.85 

A7K 

2,332,028 

0.94 

2,988,048 

0.96 

AlOA 

26,659,475 

0.88 

24,343,387 

0.87 

FlllD 

16,615,044 

0.99 

16,763,485 

0.99 

B52G 

35,791,307 

0.99 

35,378,414 

0.99 

B52H 

35,951,397 

1.00 

35,560,513 

1.00 

C5A 

50,093,889 

0.99 

51,346,253 

0.99 

C130B 

(5,829,918) 

1.12 

(4,799,561) 

1.14 

C130E 

26,439,330 

0.90 

24,240,454 

0.90 

C141B 

33,481,089 

0.93 

32,560,226 

0.93 

PBlllA 

26,765,860 

0.99 

26,156,160 

0.99 

Table  23  (Continued) 

Model  Sensitivity  Test  Results 


"THRUST"  Variable 

Multiplied 

by  1.2 

NEW 

NEW 

NEW 

EST./ 

NEW 

EST./ 

LINEAR 

ORIG, 

ARITHMETIC 

ORIG. 

MDS 

ESTIMATE 

EST. 

ESTIMATE 

EST. 

F16A 

44,823,135 

1.13 

44,443,737 

1.14 

F4D 

22,239,140 

1.19 

22,292,820 

1.21 

F4E 

25,398,441 

1.17 

25,552,156 

1.19 

A7D 

13,800,101 

1.28 

14,132,678 

1.30 

A7K 

5,515,802 

2.23 

6,434,530 

2.06 

AlOA 

32.,342,939 

1.06 

30,003,614 

1.07 

FlllD 

21,232,698 

1.26 

21,756,592 

1.28 

B52G 

38,954,921 

1.08 

38,783,811 

1.09 

B52H 

39,669,525 

1.10 

39,587,384 

1.11 

C5A 

58,953,710 

1.17 

60,950,435 

1.18 

C130B 

(3,256,703) 

0.62 

(2,082,242) 

0.49 

C130E 

31,373,665 

1.07 

29,199,033 

1.08 

C141B 

40,480,559 

1.12 

39,818,921 

1.14 

FBlllA 

31,194,512 

1.16 

30,955,520 

1.18 

"THRUST"  Variable  Multiplied  by 

.8 

NEW 

NEW 

NEW 

EST./ 

NEW 

EST./ 

LINEAR 

ORIG. 

ARITHMETIC 

ORIG. 

MDS 

ESTIMATE 

EST. 

ESTIMATE 

EST. 

F16A 

34,836,446 

0.87 

33,569,286 

0.86 

F4D 

15,114,770 

0.81 

14,535,134 

0.79 

F4E 

17,896,900 

0.83 

17,.'^83,769 

0.81 

A7D 

7,828,204 

0.72 

7,628,912 

0.70 

A7K 

(560,866) 

(0.23) 

U82,320) 

(0.06) 

AlOA 

28,543,974 

0.94 

25  .866,942 

0.93 

FlllD 

12,499,060 

0.74 

12,246,581 

0.72 

B52G 

33,192,563 

0.92 

32,509,212 

0.91 

B52H 

32,545,156 

0.90 

31,829,698 

0.09 

C5A 

41,853,129 

0.83 

42,329,706 

0.82 

C130B 

(7,191,031) 

1.38 

(6,366,310) 

1.51 

C130E 

27,439,337 

0.93 

24,914,965 

0.92 

C141B 

31,679,867 

0.88 

30,235,897 

0.86 

FBlllA 

22,666,224 

0.84 

21,663,114 

0.82 

-r 
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Table  23  (Continued) 

Model  Sensitivity  Test  Results 


TOEMPWT  and  T0EMPWT2 
Multiplied  by  1.2 


NEW 

NEW 

NEW 

EST./ 

NEW 

EST./ 

LINEAR 

ORIG. 

ARITHMETIC 

ORIG. 

MDS 

ESTIMATE 

EST. 

ESTIMATE 

EST. 

F16A 

60,945,609 

1.53 

61,293,680 

1.57 

F4D 

37,486,025 

2.01 

36,097,700 

1.96 

F4E 

40,250,368 

1.86 

38,765,765 

1.81 

A7D 

29,008,730 

2.68 

27,428,438 

2.52 

A7K 

20,480,102 

8.27 

19,325,961 

6.18 

AlOA 

51,539,598 

1.69 

50,180,931 

1.80 

FlllD 

36,312,315 

2.15 

35,904,080 

2.11 

B52G 

60,815,791 

1.69 

66,245,733 

1.86 

B52H 

60,301,595 

1.67 

64,967,814 

1,82 

C5A 

72,315,197 

1.43 

75,639,133 

1.46 

C130B 

11,849,059 

(2.27) 

10,345,571 

(2.45) 

C130E 

51,044,381 

1.74 

50,459,835 

1.86 

C141B 

56,987,700 

1.58 

56,876,972 

1.62 

FBlllA 

48,578,704 

1.80 

49,737,775 

1.89 

TOEMPWT 

and  TOEMPWT2 

Multiplied  by 

.8 

NEW 

NEW 

NEW 

EST./ 

NEW 

EST./ 

LINEAR 

ORIG. 

ARITHMETIC 

ORIG. 

MDS 

ESTIMATE 

EST. 

ESTIMATE 

EST. 

F16A 

18,713,972 

0.47 

20,771,555 

0.53 

F4D 

(132,115) 

(0.01) 

3,945,477 

0.21 

F4E 

3,044,973 

0.14 

7,315,216 

0.34 

A7D 

(7,380,425) 

(0.68) 

(2,657,276) 

(0.24) 

A7K 

(15,525,166) 

(6.27) 

(10,128,323) 

(3.24) 

AlOA 

9,347,314 

0.31 

9,734,289 

0.35 

FlllD 

(2,580,557) 

(0.15) 

1,535,910 

0.09 

B52G 

11,331,694 

0.31 

10,610,785 

0.30 

B52H 

11,913,086 

0.33 

11,769,135 

0.33 

C5A 

28,491,642 

0.57 

32,004,474 

0.62 

C130B 

(22,296,794) 

4.27 

(16,145,060) 

3.82 

C130E 

7,768,621 

0.26 

7,909,225 

0.29 

C141B 

15,172,725 

0.42 

17,150,493 

0.49 

FBlllD 

5,282,032 

0.20 

7,146,033 

0.27 
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Appendix  G.  Demand  Volatility  Analysis  Database 


The  tables  presented  in  Appendix  G  provide  the  data 
used  to  perform  the  demand  volatility  analysis.  Table  24 
and  25  provide  the  actual  data  used  in  the  analysis.  Table 
26  provides  the  original  replenishment  spares  database.  It 
is  in  then  year  dollars  and  the  engine  spares  and  common 
spares  pools  have  not  been  distributed.  Table  27  provides 
the  inflation  rates  used  to  adjust  the  figures  in  Table  26 
to  Fi'91  constant  dollars.  Table  28  shows  what  percentage  of 
the  engine  hours  were  associated  with  the  F-15  and  F-16 
aircraft  and  then  shows  how  this  data  was  used  to  determine 
how  the  engine  spares  pool  would  be  distributed  between  the 
P-15  and  F-16.  Finally,  Table  29  provides  the  common  spares 
allocation  factors  used  to  distribute  the  common  spares  pool 
among  the  MD. 

Several  sources  were  used  to  obtain  the  data.  The 
condemnations  and  engine  hours  data  was  obtained  from  the 
Weapon  System  Cost  Retrieval  System  (WSCRS).  The  orignal 
replenishment  spares  requirements  data  was  obtained  from 
unpublished  records  maintained  by  HQ  AFLC/FMBSR.  The  common 
spares  distribution  factors  were  obtained  from  Ms.  Virginia 
Mattern  of  the  Logistics  Management  Institute  (LMI). 

Finally,  the  inflation  factors  were  obtained  from  AFR  173- 
13 ,  US  Air  Force  Cost  and  Planning  Factors . 
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Table  26 


f 


Original  Replenishment  Spares  Requirements  Data 


MDS 

FY82 

FY83 

FY84 

FY85 

FY86 

FY87 

FY88 

A7 

48.2 

57.3 

28.4 

20.3 

11.8 

14.2 

9.3 

AlO 

81.9 

114.4 

101.1 

84.3 

37.8 

43.9 

46 

B52 

155.8 

275.7 

196.2 

96.9 

75.3 

36.9 

50.5 

PBlll 

37.2 

62.2 

28.3 

19.8 

10.3 

17.2 

0 

Fill 

357.4 

459 

270.5 

220.6 

155 

255.2 

232.6 

C5 

434.6 

148.9 

97 

45.4 

78.4 

124.5 

150 

C130 

123 

84.5 

93.1 

100.3 

87.4 

53.3 

51.8 

C135 

104.3 

69.2 

54.5 

113.3 

107.5 

69.1 

111.6 

C141 

53 

36.7 

43.9 

56 . 5 

16 

46.7 

34.1 

F4 

143.9 

237.7 

181.2 

87.1 

57.8 

58.1 

40.5 

F15 

284.1 

192.8 

182.8 

93.8 

62.6 

157.9 

224.4 

F16 

59.7 

61.5 

77 

313.4 

162.9 

111.6 

211.9 

T37 

30.1 

27.5 

12 

7.5 

3.5 

1.6 

T38 

77.2 

31.9 

42.9 

30.8 

15.5 

20.7 

FIDO 

409.8 

421.4 

285.4 

397.8 

453.7 

493.7 

380.9 

FllO 

0 

0 

0 

0 

66,9 

107.7 

155.8 

COMM 

704.9 

719 

419.8 

419.4 

276 

239.5 

265.3 

TOT 

3105.1 

2999.7 

2114.1 

2140.8 

1697.7 

1848.5 

1987.0 

Table  27. 

Inflation  Factors 

FY82 

FY83  FY84 

PY85 

FY86  FY87 

FY88 

,771 

.801  .824 

.852 

,886  .921 

.  959 
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Table  28 


Engine  Spares  Requirements  Distribution 


PY82 

FY83 

FY84 

FY85 

FY86 

FY87 

PY88 

F15 

298,958 

327,872 

353,232 

363,792 

389,576 

385,474 

360,181 

F15A 

F15B 

F15D 

2,907 

3,936 

F15C 

1,802 

19,855 

37,977 

F15E 

838 

SUB 

298,958 

327,872 

353,232 

363,792 

391,378 

408,236 

402,932 

F16 

95,755 

137,843 

194,490 

211,820 

242,284 

268,076 

271,992 

F16B 

Fisc 

2,442 

6,255 

F16D 

419 

299 

SUB 

95,755 

137,843 

194,490 

211,820 

242,284 

270,937 

278,546 

TOT 

394,713 

465,715 

547,722 

575,612 

633,662 

679,173 

681,478 

Fi5% 

0.76 

0.70 

0.64 

0.63 

0.62 

0.60 

0.59 

F15 

SHARE 

310.38 

296.67 

184.06 

251.41 

280.23 

296.75 

225.21 

Fie 

SHARE 

99.42 

124.73 

101.34 

146.39 

173.47 

196.95 

155.69 

LMI 

Common 

Table  29 

Spares  Distribution  Factors 

FY82 

FY83 

FY84 

FY85 

FY86 

FY87 

FY88 

A7 

3.9 

1 

1.2 

1.8 

1.0 

1.2 

2.6 

AlO 

7.0 

4.2 

3.0 

4.0 

4.0 

2.2 

0.9 

B52 

17.5 

25.2 

15.8 

22.1 

22.0 

8.7 

5.8 

FBlll 

0.6 

0.2 

0.3 

0.3 

0.6 

0.5 

0.2 

Fill 

4.6 

1.5 

1.8 

1.2 

0.7 

1.3 

1.1 

C5 

1.1 

0.7 

0.9 

1.3 

1.3 

1.5 

0.8 

C130 

3.5 

1.8 

5.1 

7.5 

7.7 

7.9 

4.4 

C135 

5.2 

11.4 

12 

11.8 

13.8 

19.0 

12.2 

C141 

10.4 

13.3 

12.3 

7.9 

9.8 

13.5 

8.1 

P4 

24.2 

10.1 

9 

5.7 

4.0 

2.8 

2.4 

F15 

6.1 

2.5 

4.1 

3.8 

2.4 

4.2 

8.6 

F16 

5.7 

3.2 

4.4 

10.1 

6.2 

13.7 

33.0 

T37 

1.4 

1.0 

0.4 

T38 

4.0 

12.7 

11.4 

1.4 

2.0 

1.5 

> 


>■ 
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